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Preface to the third edition 


As is natural, in the four years since the publication of the second edition of 
this book we have somewhat modified our views on what should be included 
and how it should be presented. In this new edition, although the range of topics 
covered has been extended, there has been no significant shift in the general level 
of difficulty or in the degree of mathematical sophistication required. Further, we 
have aimed to preserve the same style of presentation as seems to have been well 
received in the first two editions. However, a significant change has been made 
to the format of the chapters, specifically to the way that the exercises, together 
with their hints and answers, have been treated; the details of the change are 
explained below. 

The two major chapters that are new in this third edition are those dealing with 
‘special functions’ and the applications of complex variables. The former presents 
a systematic account of those functions that appear to have arisen in a more 
or less haphazard way as a result of studying particular physical situations, and 
are deemed ‘special’ for that reason. The treatment presented here shows that, 
in fact, they are nearly all particular cases of the hypergeometric or confluent 
hypergeometric functions, and are special only in the sense that the parameters 
of the relevant function take simple or related values. 

The second new chapter describes how the properties of complex variables can 
be used to tackle problems arising from the description of physical situations 
or from other seemingly unrelated areas of mathematics. To topics treated in 
earlier editions, such as the solution of Laplace’s equation in two dimensions, the 
summation of series, the location of zeros of polynomials and the calculation of 
inverse Laplace transforms, has been added new material covering Airy integrals, 
saddle-point methods for contour integral evaluation, and the WKB approach to 
asymptotic forms. 

Other new material includes a stand-alone chapter on the use of coordinate-free 
operators to establish valuable results in the field of quantum mechanics; amongst 
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the physical topics covered are angular momentum and uncertainty principles. 
There are also significant additions to the treatment of numerical integration. 
In particular, Gaussian quadrature based on Legendre, Laguerre, Hermite and 
Chebyshev polynomials is discussed, and appropriate tables of points and weights 
are provided. 

We now turn to the most obvious change to the format of the book, namely 
the way that the exercises, hints and answers are treated. The second edition of 
Mathematical Methods for Physics and Engineering carried more than twice as 
many exercises, based on its various chapters, as did the first. In its preface we 
discussed the general question of how such exercises should be treated but, in 
the end, decided to provide hints and outline answers to all problems, as in the 
first edition. This decision was an uneasy one as, on the one hand, it did not 
allow the exercises to be set as totally unaided homework that could be used for 
assessment purposes but, on the other, it did not give a full explanation of how 
to tackle a problem when a student needed explicit guidance or a model answer. 

In order to allow both of these educationally desirable goals to be achieved, 
we have, in this third edition, completely changed the way in which this matter 
is handled. A large number of exercises have been included in the penultimate 
subsections of the appropriate, sometimes reorganised, chapters. Hints and outline 
answers are given, as previously, in the final subsections, but only for the odd- 
numbered exercises. This leaves all even-numbered exercises free to be set as 
unaided homework, as described below. 

For the four hundred plus odd-numbered exercises, complete solutions are 
available, to both students and their teachers, in the form of a separate manual, 
Student Solutions Manual for Mathematical Methods for Physics and Engineering 
(Cambridge: Cambridge University Press, 2006); the hints and outline answers 
given in this main text are brief summaries of the model answers given in the 
manual. There, each original exercise is reproduced and followed by a fully 
worked solution. For those original exercises that make internal reference to this 
text or to other (even-numbered) exercises not included in the solutions manual, 
the questions have been reworded, usually by including additional information, 
so that the questions can stand alone. 

In many cases, the solution given in the manual is even fuller than one that 
might be expected of a good student that has understood the material. This is 
because we have aimed to make the solutions instructional as well as utilitarian. 
To this end, we have included comments that are intended to show how the 
plan for the solution is fomulated and have given the justifications for particular 
intermediate steps (something not always done, even by the best of students). We 
have also tried to write each individual substituted formula in the form that best 
indicates how it was obtained, before simplifying it at the next or a subsequent 
stage. Where several lines of algebraic manipulation or calculus are needed to 
obtain a final result, they are normally included in full; this should enable the 
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student to determine whether an incorrect answer is due to a misunderstanding 
of principles or to a technical error. 

The remaining four hundred or so even-numbered exercises have no hints or 
answers, outlined or detailed, available for general access. They can therefore be 
used by instructors as a basis for setting unaided homework. Full solutions to 
these exercises, in the same general format as those appearing in the manual 
(though they may contain references to the main text or to other exercises), are 
available without charge to accredited teachers as downloadable pdf files on the 
password-protected website http://www.cambridge.org/9780521679718. Teachers 
wishing to have access to the website should contact solutions@cambridge.org 
for registration details. 

In all new publications, errors and typographical mistakes are virtually un- 
avoidable, and we would be grateful to any reader who brings instances to 
our attention. Retrospectively, we would like to record our thanks to Reinhard 
Gerndt, Paul Renteln and Joe Tenn for making us aware of some errors in 
the second edition. Finally, we are extremely grateful to Dave Green for his 
considerable and continuing advice concerning IATRX. 


Ken Riley, Michael Hobson, 
Cambridge, 2006 
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Since the publication of the first edition of this book, both through teaching the 
material it covers and as a result of receiving helpful comments from colleagues, 
we have become aware of the desirability of changes in a number of areas. 
The most important of these is that the mathematical preparation of current 
senior college and university entrants is now less thorough than it used to be. 
To match this, we decided to include a preliminary chapter covering areas such 
as polynomial equations, trigonometric identities, coordinate geometry, partial 
fractions, binomial expansions, necessary and sufficient condition and proof by 
induction and contradiction. 

Whilst the general level of what is included in this second edition has not 
been raised, some areas have been expanded to take in topics we now feel were 
not adequately covered in the first. In particular, increased attention has been 
given to non-square sets of simultaneous linear equations and their associated 
matrices. We hope that this more extended treatment, together with the inclusion 
of singular value matrix decomposition, will make the material of more practical 
use to engineering students. In the same spirit, an elementary treatment of linear 
recurrence relations has been included. The topic of normal modes has been given 
a small chapter of its own, though the links to matrices on the one hand, and to 
representation theory on the other, have not been lost. 

Elsewhere, the presentation of probability and statistics has been reorganised to 
give the two aspects more nearly equal weights. The early part of the probability 
chapter has been rewritten in order to present a more coherent development 
based on Boolean algebra, the fundamental axioms of probability theory and 
the properties of intersections and unions. Whilst this is somewhat more formal 
than previously, we think that it has not reduced the accessibility of these topics 
and hope that it has increased it. The scope of the chapter has been somewhat 
extended to include all physically important distributions and an introduction to 
cumulants. 
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Statistics now occupies a substantial chapter of its own, one that includes sys- 
tematic discussions of estimators and their efficiency, sample distributions and t- 
and F-tests for comparing means and variances. Other new topics are applications 
of the chi-squared distribution, maximum-likelihood parameter estimation and 
least-squares fitting. In other chapters we have added material on the following 
topics: curvature, envelopes, curve-sketching, more refined numerical methods 
for differential equations and the elements of integration using Monte Carlo 
techniques. 

Over the last four years we have received somewhat mixed feedback about 
the number of exercises at the ends of the various chapters. After consideration, 
we decided to increase the number substantially, partly to correspond to the 
additional topics covered in the text but mainly to give both students and 
their teachers a wider choice. There are now nearly 800 such exercises, many with 
several parts. An even more vexed question has been whether to provide hints and 
answers to all the exercises or just to ‘the odd-numbered’ ones, as is the normal 
practice for textbooks in the United States, thus making the remainder more 
suitable for setting as homework. In the end, we decided that hints and outline 
solutions should be provided for all the exercises, in order to facilitate independent 
study while leaving the details of the calculation as a task for the student. 

In conclusion, we hope that this edition will be thought by its users to be 
‘heading in the right direction’ and would like to place on record our thanks to 
all who have helped to bring about the changes and adjustments. Naturally, those 
colleagues who have noted errors or ambiguities in the first edition and brought 
them to our attention figure high on the list, as do the staff at The Cambridge 
University Press. In particular, we are grateful to Dave Green for continued I4TBxX 
advice, Susan Parkinson for copy-editing the second edition with her usual keen 
eye for detail and flair for crafting coherent prose and Alison Woollatt for once 
again turning our basic IATsX into a beautifully typeset book. Our thanks go 
to all of them, though of course we accept full responsibility for any remaining 
errors or ambiguities, of which, as with any new publication, there are bound to 
be some. 

On a more personal note, KFR again wishes to thank his wife Penny for her 
unwavering support, not only in his academic and tutorial work, but also in their 
joint efforts to convert time at the bridge table into ‘green points’ on their record. 
MPH is once more indebted to his wife, Becky, and his mother, Pat, for their 
tireless support and encouragement above and beyond the call of duty. MPH 
dedicates his contribution to this book to the memory of his father, Ronald 
Leonard Hobson, whose gentle kindness, patient understanding and unbreakable 
spirit made all things seem possible. 


Ken Riley, Michael Hobson 
Cambridge, 2002 
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A knowledge of mathematical methods is important for an increasing number of 
university and college courses, particularly in physics, engineering and chemistry, 
but also in more general science. Students embarking on such courses come from 
diverse mathematical backgrounds, and their core knowledge varies considerably. 
We have therefore decided to write a textbook that assumes knowledge only of 
material that can be expected to be familiar to all the current generation of 
students starting physical science courses at university. In the United Kingdom 
this corresponds to the standard of Mathematics A-level, whereas in the United 
States the material assumed is that which would normally be covered at junior 
college. 

Starting from this level, the first six chapters cover a collection of topics 
with which the reader may already be familiar, but which are here extended 
and applied to typical problems encountered by first-year university students. 
They are aimed at providing a common base of general techniques used in 
the development of the remaining chapters. Students who have had additional 
preparation, such as Further Mathematics at A-level, will find much of this 
material straightforward. 

Following these opening chapters, the remainder of the book is intended to 
cover at least that mathematical material which an undergraduate in the physical 
sciences might encounter up to the end of his or her course. The book is also 
appropriate for those beginning graduate study with a mathematical content, and 
naturally much of the material forms parts of courses for mathematics students. 
Furthermore, the text should provide a useful reference for research workers. 

The general aim of the book is to present a topic in three stages. The first 
stage is a qualitative introduction, wherever possible from a physical point of 
view. The second is a more formal presentation, although we have deliberately 
avoided strictly mathematical questions such as the existence of limits, uniform 
convergence, the interchanging of integration and summation orders, etc. on the 
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grounds that ‘this is the real world; it must behave reasonably’. Finally a worked 
example is presented, often drawn from familiar situations in physical science 
and engineering. These examples have generally been fully worked, since, in 
the authors’ experience, partially worked examples are unpopular with students. 
Only in a few cases, where trivial algebraic manipulation is involved, or where 
repetition of the main text would result, has an example been left as an exercise 
for the reader. Nevertheless, a number of exercises also appear at the end of each 
chapter, and these should give the reader ample opportunity to test his or her 
understanding. Hints and answers to these exercises are also provided. 

With regard to the presentation of the mathematics, it has to be accepted that 
many equations (especially partial differential equations) can be written more 
compactly by using subscripts, e.g. u,, for a second partial derivative, instead of 
the more familiar 4*u/@xdy, and that this certainly saves typographical space. 
However, for many students, the labour of mentally unpacking such equations 
is sufficiently great that it is not possible to think of an equation’s physical 
interpretation at the same time. Consequently, wherever possible we have decided 
to write out such expressions in their more obvious but longer form. 

During the writing of this book we have received much help and encouragement 
from various colleagues at the Cavendish Laboratory, Clare College, Trinity Hall 
and Peterhouse. In particular, we would like to thank Peter Scheuer, whose 
comments and general enthusiasm proved invaluable in the early stages. For 
reading sections of the manuscript, for pointing out misprints and for numerous 
useful comments, we thank many of our students and colleagues at the University 
of Cambridge. We are especially grateful to Chris Doran, John Huber, Garth 
Leder, Tom K6rner and, not least, Mike Stobbs, who, sadly, died before the book 
was completed. We also extend our thanks to the University of Cambridge and 
the Cavendish teaching staff, whose examination questions and lecture hand-outs 
have collectively provided the basis for some of the examples included. Of course, 
any errors and ambiguities remaining are entirely the responsibility of the authors, 
and we would be most grateful to have them brought to our attention. 

We are indebted to Dave Green for a great deal of advice concerning typesetting 
in I4TRX and to Andrew Lovatt for various other computing tips. Our thanks 
also go to Anja Visser and Graga Rocha for enduring many hours of (sometimes 
heated) debate. At Cambridge University Press, we are very grateful to our editor 
Adam Black for his help and patience and to Alison Woollatt for her expert 
typesetting of such a complicated text. We also thank our copy-editor Susan 
Parkinson for many useful suggestions that have undoubtedly improved the style 
of the book. 

Finally, on a personal note, KFR wishes to thank his wife Penny, not only for 
a long and happy marriage, but also for her support and understanding during 
his recent illness — and when things have not gone too well at the bridge table! 
MPH is indebted both to Rebecca Morris and to his parents for their tireless 
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support and patience, and for their unending supplies of tea. SJB is grateful to 
Anthony Gritten for numerous relaxing discussions about J.S. Bach, to Susannah 
Ticciati for her patience and understanding, and to Kate Isaak for her calming 
late-night e-mails from the USA. 


Ken Riley, Michael Hobson and Stephen Bence 
Cambridge, 1997 
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Preliminary algebra 


This opening chapter reviews the basic algebra of which a working knowledge is 
presumed in the rest of the book. Many students will be familiar with much, if 
not all, of it, but recent changes in what is studied during secondary education 
mean that it cannot be taken for granted that they will already have a mastery 
of all the topics presented here. The reader may assess which areas need further 
study or revision by attempting the exercises at the end of the chapter. The main 
areas covered are polynomial equations and the related topic of partial fractions, 
curve sketching, coordinate geometry, trigonometric identities and the notions of 
proof by induction or contradiction. 


1.1 Simple functions and equations 


It is normal practice when starting the mathematical investigation of a physical 
problem to assign an algebraic symbol to the quantity whose value is sought, either 
numerically or as an explicit algebraic expression. For the sake of definiteness, in 
this chapter we will use x to denote this quantity most of the time. Subsequent 
steps in the analysis involve applying a combination of known laws, consistency 
conditions and (possibly) given constraints to derive one or more equations 
satisfied by x. These equations may take many forms, ranging from a simple 
polynomial equation to, say, a partial differential equation with several boundary 
conditions. Some of the more complicated possibilities are treated in the later 
chapters of this book, but for the present we will be concerned with techniques 
for the solution of relatively straightforward algebraic equations. 


1.1.1 Polynomials and polynomial equations 


Firstly we consider the simplest type of equation, a polynomial equation, in which 
a polynomial expression in x, denoted by f(x), is set equal to zero and thereby 
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forms an equation which is satisfied by particular values of x, called the roots of 
the equation: 


F(X) = ayx” + Gyax" | +--+ + a1x +a =0. (1.1) 


Here n is an integer > 0, called the degree of both the polynomial and the 
equation, and the known coefficients ao, a1,...,d, are real quantities with a, # 0. 

Equations such as (1.1) arise frequently in physical problems, the coefficients a; 
being determined by the physical properties of the system under study. What is 
needed is to find some or all of the roots of (1.1), ie. the x-values, 0, that satisfy 
f(a) = 0; here k is an index that, as we shall see later, can take up to n different 
values, 1e. k = 1,2,...,n. The roots of the polynomial equation can equally well 
be described as the zeros of the polynomial. When they are real, they correspond 
to the points at which a graph of f(x) crosses the x-axis. Roots that are complex 
(see chapter 3) do not have such a graphical interpretation. 

For polynomial equations containing powers of x greater than x* general 
methods do not exist for obtaining explicit expressions for the roots o,. Even 
for n = 3 and n = 4 the prescriptions for obtaining the roots are sufficiently 
complicated that it is usually preferable to obtain exact or approximate values 
by other methods. Only for n = 1 and n = 2 can closed-form solutions be given. 
These results will be well known to the reader, but they are given here for the 
sake of completeness. For n = 1, (1.1) reduces to the linear equation 


ax +a) =0; (1.2) 
the solution (root) is «) = —do/a,. For n = 2, (1.1) reduces to the quadratic 
equation 

ax? +ayx +a) =0; (1.3) 


the two roots a, and a are given by 


—a, + \/at — 4ayao ‘is 


ao= 
12 Fas 
When discussing specifically quadratic equations, as opposed to more general 


polynomial equations, it is usual to write the equation in one of the two notations 
ax? +bx +c =0, ax? +2bx +c =0, (1.5) 


with respective explicit pairs of solutions 


—b+ /b2 — 4ac —b+ Jb? — ac 
2a ; a y 


u2= 12 = (1.6) 


Of course, these two notations are entirely equivalent and the only important 
point is to associate each form of answer with the corresponding form of equation; 
most people keep to one form, to avoid any possible confusion. 
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If the value of the quantity appearing under the square root sign is positive 
then both roots are real; if it is negative then the roots form a complex conjugate 
pair, ic. they are of the form p+iq with p and q real (see chapter 3); if it has 
zero value then the two roots are equal and special considerations usually arise. 

Thus linear and quadratic equations can be dealt with in a cut-and-dried way. 
We now turn to methods for obtaining partial information about the roots of 
higher-degree polynomial equations. In some circumstances the knowledge that 
an equation has a root lying in a certain range, or that it has no real roots at all, 
is all that is actually required. For example, in the design of electronic circuits 
it is necessary to know whether the current in a proposed circuit will break 
into spontaneous oscillation. To test this, it is sufficient to establish whether a 
certain polynomial equation, whose coefficients are determined by the physical 
parameters of the circuit, has a root with a positive real part (see chapter 3); 
complete determination of all the roots is not needed for this purpose. If the 
complete set of roots of a polynomial equation is required, it can usually be 
obtained to any desired accuracy by numerical methods such as those described 
in chapter 27. 

There is no explicit step-by-step approach to finding the roots of a general 
polynomial equation such as (1.1). In most cases analytic methods yield only 
information about the roots, rather than their exact values. To explain the relevant 
techniques we will consider a particular example, ‘thinking aloud’ on paper and 
expanding on special points about methods and lines of reasoning. In more 
routine situations such comment would be absent and the whole process briefer 
and more tightly focussed. 


Example: the cubic case 


Let us investigate the roots of the equation 
g(x) = 4x? + 3x? — 6x —1=0 (1.7) 


or, in an alternative phrasing, investigate the zeros of g(x). We note first of all 
that this is a cubic equation. It can be seen that for x large and positive g(x) 
will be large and positive and, equally, that for x large and negative g(x) will 
be large and negative. Therefore, intuitively (or, more formally, by continuity) 
g(x) must cross the x-axis at least once and so g(x) = 0 must have at least one 
real root. Furthermore, it can be shown that if f(x) is an nth-degree polynomial 
then the graph of f(x) must cross the x-axis an even or odd number of times 
as x varies between —oo and +o0, according to whether n itself is even or odd. 
Thus a polynomial of odd degree always has at least one real root, but one of 
even degree may have no real root. A small complication, discussed later in this 
section, occurs when repeated roots arise. 

Having established that g(x) = 0 has at least one real root, we may ask how 
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many real roots it could have. To answer this we need one of the fundamental 
theorems of algebra, mentioned above: 


An nth-degree polynomial equation has exactly n roots. 


It should be noted that this does not imply that there are n real roots (only that 
there are not more than n); some of the roots may be of the form p + iq. 

To make the above theorem plausible and to see what is meant by repeated 
roots, let us suppose that the nth-degree polynomial equation f(x) = 0, (1.1), has 
r roots 01, 0%2,...,%,, considered distinct for the moment. That is, we suppose that 
f(x) =0 for k = 1,2,...,7r, so that f(x) vanishes only when x is equal to one of 
the r values «%. But the same can be said for the function 


F(x) = A(x — 01 )(x — a)- ++ (x — a), (1.8) 


in which A is a non-zero constant; F(x) can clearly be multiplied out to form a 
polynomial expression. 

We now call upon a second fundamental result in algebra: that if two poly- 
nomial functions f(x) and F(x) have equal values for all values of x, then their 
coefficients are equal on a term-by-term basis. In other words, we can equate 
the coefficients of each and every power of x in the two expressions (1.8) and 
(1.1); in particular we can equate the coefficients of the highest power of x. From 
this we have Ax" = a,x" and thus that r = n and A = a,. As r is both equal 
to n and to the number of roots of f(x) = 0, we conclude that the nth-degree 
polynomial f(x) = 0 has n roots. (Although this line of reasoning may make the 
theorem plausible, it does not constitute a proof since we have not shown that it 
is permissible to write f(x) in the form of equation (1.8).) 

We next note that the condition f(a) =0 for k = 1,2,...,r, could also be met 
if (1.8) were replaced by 


F(x) = A(x — a1 )""(x = a) ++ + (0 = oO), (1.9) 


with A = a,. In (1.9) the m, are integers > 1 and are known as the multiplicities 
of the roots, m, being the multiplicity of «,. Expanding the right-hand side (RHS) 
leads to a polynomial of degree m; +m +----+-m,. This sum must be equal to n. 
Thus, if any of the m, is greater than unity then the number of distinct roots, r, 
is less than n; the total number of roots remains at n, but one or more of the % 
counts more than once. For example, the equation 


F(x) = A(x — o41)°(x — a2)°(x — a3)(x — a4) = 0 


has exactly seven roots, « being a double root and « a triple root, whilst «3 and 
a4 are unrepeated (simple) roots. 

We can now say that our particular equation (1.7) has either one or three real 
roots but in the latter case it may be that not all the roots are distinct. To decide 
how many real roots the equation has, we need to anticipate two ideas from the 
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Figure 1.1 Two curves (x) and ¢2(x), both with zero derivatives at the 
same values of x, but with different numbers of real solutions to ;(x) = 0. 


next chapter. The first of these is the notion of the derivative of a function, and 
the second is a result known as Rolle’s theorem. 

The derivative f'(x) of a function f(x) measures the slope of the tangent to 
the graph of f(x) at that value of x (see figure 2.1 in the next chapter). For 
the moment, the reader with no prior knowledge of calculus is asked to accept 
that the derivative of ax” is nax"~!, so that the derivative g/(x) of the curve 
g(x) = 4x3 + 3x? — 6x — 1 is given by g’(x) = 12x? + 6x — 6. Similar expressions 
for the derivatives of other polynomials are used later in this chapter. 

Rolle’s theorem states that if f(x) has equal values at two different values of x 
then at some point between these two x-values its derivative is equal to zero; Le. 
the tangent to its graph is parallel to the x-axis at that point (see figure 2.2). 

Having briefly mentioned the derivative of a function and Rolle’s theorem, we 
now use them to establish whether g(x) has one or three real zeros. If g(x) = 0 
does have three real roots o, Le. g(a.) = 0 for k = 1,2,3, then it follows from 
Rolle’s theorem that between any consecutive pair of them (say « and a2) there 
must be some real value of x at which g’(x) = 0. Similarly, there must be a further 
zero of g’(x) lying between «2 and «3. Thus a necessary condition for three real 
roots of g(x) = 0 is that g/(x) = 0 itself has two real roots. 

However, this condition on the number of roots of g/(x) = 0, whilst necessary, 
is not sufficient to guarantee three real roots of g(x) = 0. This can be seen by 
inspecting the cubic curves in figure 1.1. For each of the two functions $;(x) and 
2(x), the derivative is equal to zero at both x = f; and x = fp. Clearly, though, 
2(x) = 0 has three real roots whilst ¢;(x) = 0 has only one. It is easy to see that 
the crucial difference is that #1(f1) and ¢)(f2) have the same sign, whilst #2(f1) 
and ¢2(f2) have opposite signs. 

It will be apparent that for some equations, $(x) = 0 say, (x) equals zero 
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at a value of x for which ¢(x) is also zero. Then the graph of (x) just touches 
the x-axis. When this happens the value of x so found is, in fact, a double real 
root of the polynomial equation (corresponding to one of the m, in (1.9) having 
the value 2) and must be counted twice when determining the number of real 
roots. 

Finally, then, we are in a position to decide the number of real roots of the 
equation 


g(x) = 4x? + 3x? — 6x —1=0. 


The equation g’(x) = 0, with g’(x) = 12x? + 6x — 6, is a quadratic equation with 
explicit solutions’ 


8 —34V9+72 
12 
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so that 6; = —1 and fy = 5. The corresponding values of g(x) are g((1) = 4 and 
g(B2) = —4, which are of opposite sign. This indicates that 4x3 + 3x?—6x—1 =0 
has three real roots, one lying in the range —1 < x < 5 and the others one on 
each side of that range. 

The techniques we have developed above have been used to tackle a cubic 
equation, but they can be applied to polynomial equations f(x) = 0 of degree 
greater than 3. However, much of the analysis centres around the equation 
f'(x) = 0 and this itself, being then a polynomial equation of degree 3 or more, 
either has no closed-form general solution or one that is complicated to evaluate. 
Thus the amount of information that can be obtained about the roots of f(x) = 0 
is correspondingly reduced. 


A more general case 


To illustrate what can (and cannot) be done in the more general case we now 
investigate as far as possible the real roots of 


f(x) =x! +5x° 4x4 - 3 +? -2=0. 
The following points can be made. 


(i) This is a seventh-degree polynomial equation; therefore the number of 
real roots is 1, 3, 5 or 7. 

(ii) f(0) is negative whilst f(0o) = +00, so there must be at least one positive 
root. 


S The two roots B1, Bz are written as 612. By convention f; refers to the upper symbol in +, f2 to 
the lower symbol. 
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(iii) The equation f’(x) = 0 can be written as x(7x° + 30x4 + 4x? — 3x + 2) =0 
and thus x = 0 is a root. The derivative of f(x), denoted by f”(x), equals 
42x + 150x4 + 12x? — 6x +2. That f(x) is zero whilst f”(x) is positive 
at x = 0 indicates (subsection 2.1.8) that f(x) has a minimum there. This, 
together with the facts that f(0) is negative and f(co) = 0, implies that 
the total number of real roots to the right of x = 0 must be odd. Since 
the total number of real roots must be odd, the number to the left must 
be even (0, 2, 4 or 6). 


This is about all that can be deduced by simple analytic methods in this case, 
although some further progress can be made in the ways indicated in exercise 1.3. 

There are, in fact, more sophisticated tests that examine the relative signs of 
successive terms in an equation such as (1.1), and in quantities derived from 
them, to place limits on the numbers and positions of roots. But they are not 
prerequisites for the remainder of this book and will not be pursued further 
here. 

We conclude this section with a worked example which demonstrates that the 
practical application of the ideas developed so far can be both short and decisive. 


> For what values of k, if any, does 
f(x) = x3 — 3x7 + 6x+k =0 


have three real roots? 


Firstly we study the equation f’(x) = 0, ie. 3x7 — 6x + 6 = 0. This is a quadratic equation 
but, using (1.6), because 67 < 4 x 3 x 6, it can have no real roots. Therefore, it follows 
immediately that f(x) has no maximum or minimum; consequently f(x) = 0 cannot have 
more than one real root, whatever the value of k. < 


1.1.2 Factorising polynomials 


In the previous subsection we saw how a polynomial with r given distinct zeros 
% could be constructed as the product of factors containing those zeros: 


F(X) = n(x — 1) (% — 01) + ++ (2% — 
= anx" + dyp_1x" | +--+ ayx + a0, (1.10) 
with m; +m: +---+m, =n, the degree of the polynomial. It will cause no loss of 
generality in what follows to suppose that all the zeros are simple, i.e. all m = 1 
and r =n, and this we will do. 
Sometimes it is desirable to be able to reverse this process, in particular when 
one exact zero has been found by some method and the remaining zeros are to 


be investigated. Suppose that we have located one zero, «; it is then possible to 
write (1.10) as 


f(x) = (x — &) fi (x), (1.11) 
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where f{(x) is a polynomial of degree n—1. How can we find f1(x)? The procedure 
is much more complicated to describe in a general form than to carry out for 
an equation with given numerical coefficients a;. If such manipulations are too 
complicated to be carried out mentally, they could be laid out along the lines of 
an algebraic ‘long division’ sum. However, a more compact form of calculation 
is as follows. Write f(x) as 


F100) = Dpax"! + By ax"? + Dy—gx" 9 +++ + bx + bo. 


Substitution of this form into (1.11) and subsequent comparison of the coefficients 
of x? for p =n, n—1,..., 1, 0 with those in the second line of (1.10) generates 
the series of equations 


bn-1 = Gn, 
bn—2 — &bn—1 = Gn-1, 


by-3 oD y-2 = dn-2, 


> 
Co 
| 
g 
SS 
ll 
2 


These can be solved successively for the b;, starting either from the top or from 
the bottom of the series. In either case the final equation used serves as a check; 
if it is not satisfied, at least one mistake has been made in the computation — 
or « is not a zero of f(x) = 0. We now illustrate this procedure with a worked 
example. 


> Determine by inspection the simple roots of the equation 


f(x) = 3x4 — x3 — 10x? — 2x +4=0 


and hence, by factorisation, find the rest of its roots. 


From the pattern of coefficients it can be seen that x = —1 is a solution to the equation. 
We therefore write 


f(x) = (x + 1)(b3x? + box? + bix + bo), 


where 
b3 = 3, 
by +b3 =—1, 
b, +b, = —10, 
bo + by = —2, 
bop =4 


These equations give b3 = 3, bo 4, by 6, bo = 4 (check) and so 
f(x) = (x + Dfi(x) = (x + 13x? — 4x? — 6x + 4). 


1.1 SIMPLE FUNCTIONS AND EQUATIONS 


We now note that fi(x) = 0 if x is set equal to 2. Thus x — 2 is a factor of f1(x), which 
therefore can be written as 


f(x) = (x — 2)fo(x) = (x — 2)(eox” + c1x + €0) 


with 
c = 3, 
cy — 2c) = —4, 
Co — 2c) = —6, 
—2co = 4. 


These equations determine f2(x) as 3x? + 2x — 2. Since f(x) = 0 is a quadratic equation, 
its solutions can be written explicitly as 
—-l+y1+6 
3 : 


Thus the four roots of f(x) =0 are —1,2, +(-1 + J7) and (1 _ V7). < 


x= 


1.1.3 Properties of roots 


From the fact that a polynomial equation can be written in any of the alternative 
forms 

f(X) = apx" + yx" | +--+ + a,x + a9 =0, 

F(X) = an(x — 011)" (2% — at) +++ (% — OG )™ = O, 

F(X) = an(X — 041 )(% — 0) +++ (% — On) = 0, 
it follows that it must be possible to express the coefficients a; in terms of the 


roots 0. To take the most obvious example, comparison of the constant terms 
(formally the coefficient of x°) in the first and third expressions shows that 


An(—0 (12) «++ (—%n) = ao, 


or, using the product notation, 


iy 2 
an 


ok = ( (1.12) 
k=1 
Only slightly less obvious is a result obtained by comparing the coefficients of 


x" in the same two expressions of the polynomial: 


Soy = — (1.13) 
an 
Comparing the coefficients of other powers of x yields further results, though 
they are of less general use than the two just given. One such, which the reader 
may wish to derive, is 
n 


SS a = = (1.14) 


j=l k>j 
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In the case of a quadratic equation these root properties are used sufficiently 
often that they are worth stating explicitly, as follows. If the roots of the quadratic 
equation ax? + bx +c =0 are a and a then 


b 
1 + =—-, 
a 


Cc 
102 = -. 
a 


If the alternative standard form for the quadratic is used, b is replaced by 2b in 
both the equation and the first of these results. 


& Find a cubic equation whose roots are —4,3 and 5. 


From results (1.12) — (1.14) we can compute that, arbitrarily setting a; = 1, 


3 3. 3 
=a, = >> % =4, ay =S0 >) aja =-—17, ay = (-1)) |] % = 60. 
k=1 


j=l kj k=1 


2 


Thus a possible cubic equation is x* + (—4)x? + (—17)x + (60) = 0. Of course, any multiple 
of x3 — 4x? — 17x + 60 = 0 will do just as well. < 


1.2 Trigonometric identities 


So many of the applications of mathematics to physics and engineering are 
concerned with periodic, and in particular sinusoidal, behaviour that a sure and 
ready handling of the corresponding mathematical functions is an essential skill. 
Even situations with no obvious periodicity are often expressed in terms of 
periodic functions for the purposes of analysis. Later in this book whole chapters 
are devoted to developing the techniques involved, but as a necessary prerequisite 
we here establish (or remind the reader of) some standard identities with which he 
or she should be fully familiar, so that the manipulation of expressions containing 
sinusoids becomes automatic and reliable. So as to emphasise the angular nature 
of the argument of a sinusoid we will denote it in this section by 0 rather than x. 


1.2.1 Single-angle identities 


We give without proof the basic identity satisfied by the sinusoidal functions sin 0 
and cos 0, namely 


cos? 6 + sin? 0 = 1. (1.15) 


If sin@ and cos @ have been defined geometrically in terms of the coordinates of 
a point on a circle, a reference to the name of Pythagoras will suffice to establish 
this result. If they have been defined by means of series (with 0 expressed in 
radians) then the reader should refer to Euler’s equation (3.23) on page 93, and 
note that e” has unit modulus if @ is real. 
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Figure 1.2 Illustration of the compound-angle identities. Refer to the main 
text for details. 


Other standard single-angle formulae derived from (1.15) by dividing through 
by various powers of sin@ and cos @ are 


1+ tan? 0 = sec’ 0, (1.16) 
cot? 0+ 1 =cosec 70. (1.17) 


1.2.2 Compound-angle identities 


The basis for building expressions for the sinusoidal functions of compound 
angles are those for the sum and difference of just two angles, since all other 
cases can be built up from these, in principle. Later we will see that a study of 
complex numbers can provide a more efficient approach in some cases. 

To prove the basic formulae for the sine and cosine of a compound angle 
A+B in terms of the sines and cosines of A and B, we consider the construction 
shown in figure 1.2. It shows two sets of axes, Oxy and Ox’y’, with a common 
origin but rotated with respect to each other through an angle A. The point 
P lies on the unit circle centred on the common origin O and has coordinates 
cos(A + B),sin(A + B) with respect to the axes Oxy and coordinates cos B, sin B 
with respect to the axes Ox’y’. 

Parallels to the axes Oxy (dotted lines) and Ox'y’ (broken lines) have been 
drawn through P. Further parallels (MR and RN) to the Ox’y’ axes have been 
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drawn through R, the point (0,sin(A + B)) in the Oxy system. That all the angles 
marked with the symbol e are equal to A follows from the simple geometry of 
right-angled triangles and crossing lines. 
We now determine the coordinates of P in terms of lengths in the figure, 
expressing those lengths in terms of both sets of coordinates: 
(i) cosB =x'=TN+NP =MR+NP 
= ORsinA + RP cosA = sin(A + B)sin A + cos(A + B)cos A; 
(ii) sin B= y’ =OM—TM=OM—NR 
= ORcosA— RP sin A = sin(A + B)cos A — cos(A + B) sin A. 


Now, if equation (i) is multiplied by sin A and added to equation (ii) multiplied 
by cos A, the result is 
sin Acos B + cos Asin B = sin(A + B)(sin? A + cos’ A) = sin(A + B). 


Similarly, if equation (ii) is multiplied by sin A and subtracted from equation (i) 
multiplied by cos A, the result is 


cos A cos B — sin Asin B = cos(A + B)(cos” A + sin? A) = cos(A + B). 


Corresponding graphically based results can be derived for the sines and cosines 
of the difference of two angles; however, they are more easily obtained by setting 
B to —B in the previous results and remembering that sin B becomes — sin B 
whilst cos B is unchanged. The four results may be summarised by 


sin(A + B) = sinAcos B + cosA sin B (1.18) 
cos(A + B) = cosA cos B + sinA sin B. (1.19) 


Standard results can be deduced from these by setting one of the two angles 
equal to z or to 2/2: 


sin(z — 0) = sin 0, cos(z — 0) = —cos8@, (1.20) 
sin($x—0) =cos0, cos (4x—0) =sin0, (1.21) 


From these basic results many more can be derived. An immediate deduction, 
obtained by taking the ratio of the two equations (1.18) and (1.19) and then 
dividing both the numerator and denominator of this ratio by cos A cos B, is 


tanA + tanB 


tan(A + B) = ————— 
an(A + B) 1+ tanAtanB 


(1.22) 


One application of this result is a test for whether two lines on a graph 
are orthogonal (perpendicular); more generally, it determines the angle between 
them. The standard notation for a straight-line graph is y = mx +c, in which m 
is the slope of the graph and c is its intercept on the y-axis. It should be noted 
that the slope m is also the tangent of the angle the line makes with the x-axis. 
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Consequently the angle 0;2 between two such straight-line graphs is equal to the 
difference in the angles they individually make with the x-axis, and the tangent 
of that angle is given by (1.22): 


tan0;—tan0. — m—m 


tan > = 
oe 1+ tan 0; tan 02 1+mm 


(1.23) 


For the lines to be orthogonal we must have 0,2 = 2/2, i.e. the final fraction on 
the RHS of the above equation must equal oo, and so 


mm = —1. (1.24) 


A kind of inversion of equations (1.18) and (1.19) enables the sum or difference 
of two sines or cosines to be expressed as the product of two sinusoids; the 
procedure is typified by the following. Adding together the expressions given by 
(1.18) for sin(A + B) and sin(A — B) yields 


sin(A + B) + sin(A — B) = 2sinA cos B. 


If we now write A+ B = C and A— B = D, this becomes 


sinC 4 sinD = 2sin (<S")) cos (<S>). (1.25) 


2 2 


In a similar way each of the following equations can be derived: 


sin C sinD = 20s (S?)) sin (TS), (1.26) 
D —D 

cosC 4 cos D = 2e0s (SS ) 008 (S , (1.27) 

cos C — 00s D = ~2sin (*) sin(S*), (1.28) 


The minus sign on the right of the last of these equations should be noted; it may 
help to avoid overlooking this ‘oddity’ to recall that if C > D then cosC <cosD. 


1.2.3 Double- and half-angle identities 


Double-angle and half-angle identities are needed so often in practical calculations 
that they should be committed to memory by any physical scientist. They can be 
obtained by setting B equal to A in results (1.18) and (1.19). When this is done, 
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and use made of equation (1.15), the following results are obtained: 


sin 20 = 2 sin @ cos 90, (1.29) 
cos 20 = cos* 0 — sin? 0 
=2cos?0—1 
= 1—2sin’ 0, (1.30) 
2tan0 
tan 20 = ——_. 1.31 
an 1—tan20 Gp) 


A further set of identities enables sinusoidal functions of @ to be expressed in 
terms of polynomial functions of a variable t = tan(0/2). They are not used in 
their primary role until the next chapter, but we give a derivation of them here 
for reference. 

If t = tan(0/2), then it follows from (1.16) that 1+? = sec?(0/2) and cos(0/2) = 
(1+¢)-!/?, whilst sin(@/2) = t(1 + 7)-'/?. Now, using (1.29) and (1.30), we may 
write: 


sin = 2sin 5, cos 5 = —_ (1.32) 
0 @ 1-? 
cos 0 = cos” 5 sin? hale rR (1.33) 
2t 
tand = ce (1.34) 


It can be further shown that the derivative of 0 with respect to t takes the 
algebraic form 2/(1 + t*). This completes a package of results that enables 
expressions involving sinusoids, particularly when they appear as integrands, to 
be cast in more convenient algebraic forms. The proof of the derivative property 
and examples of use of the above results are given in subsection (2.2.7). 

We conclude this section with a worked example which is of such a commonly 
occurring form that it might be considered a standard procedure. 


> Solve for 0 the equation 
asind + bcos@ =k, 


where a,b and k are given real quantities. 


To solve this equation we make use of result (1.18) by setting a = K cos ¢ and b= K sing 
for suitable values of K and @. We then have 


k =Kcos¢sin@+K sin ¢cos 0 = K sin(0 + @), 
with 
b 
Ke=@ +h and go =tan! -. 
a 


Whether ¢ lies in 0 < @ < a or in —z < ¢ < 0 has to be determined by the individual 
signs of a and b. The solution is thus 


0 =sin! (x) —¢, 
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with K and ¢ as given above. Notice that the inverse sine yields two values in the range 0 
to 2x and that there is no real solution to the original equation if |k| > |K| = (a?+b’)!/?. 


1.3 Coordinate geometry 


We have already mentioned the standard form for a straight-line graph, namely 
y=mx+ec, (1.35) 


representing a linear relationship between the independent variable x and the 
dependent variable y. The slope m is equal to the tangent of the angle the line 
makes with the x-axis whilst c is the intercept on the y-axis. 

An alternative form for the equation of a straight line is 


ax+by+k=0, (1.36) 


to which (1.35) is clearly connected by 
m=—-—— and c=--. 


This form treats x and y on a more symmetrical basis, the intercepts on the two 
axes being —k/a and —k/b respectively. 

A power relationship between two variables, i.e. one of the form y = Ax”, can 
also be cast into straight-line form by taking the logarithms of both sides. Whilst 
it is normal in mathematical work to use natural logarithms (to base e, written 
Inx), for practical investigations logarithms to base 10 are often employed. In 
either case the form is the same, but it needs to be remembered which has been 
used when recovering the value of A from fitted data. In the mathematical (base 
e) form, the power relationship becomes 


Iny =nlnx+InA. (1.37) 


Now the slope gives the power n, whilst the intercept on the Iny axis is In A, 
which yields A, either by exponentiation or by taking antilogarithms. 

The other standard coordinate forms of two-dimensional curves that students 
should know and recognise are those concerned with the conic sections — so called 
because they can all be obtained by taking suitable sections across a (double) 
cone. Because the conic sections can take many different orientations and scalings 
their general form is complex, 


Ax? + By? + Cxy + Dx + Ey + F =0, (1.38) 


but each can be represented by one of four generic forms, an ellipse, a parabola, a 
hyperbola or, the degenerate form, a pair of straight lines. If they are reduced to 
their standard representations, in which axes of symmetry are made to coincide 
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with the coordinate axes, the first three take the forms 


(xa? | BP _, 


ae Be (ellipse), (1.39) 
(y — BY? = 4a(x — a) (parabola), (1.40) 
oe. 7 ve =1 (hyperbola). (1.41) 


Here, (a, f) gives the position of the ‘centre’ of the curve, usually taken as 
the origin (0,0) when this does not conflict with any imposed conditions. The 
parabola equation given is that for a curve symmetric about a line parallel to 
the x-axis. For one symmetrical about a parallel to the y-axis the equation would 
read (x — a)* = 4a(y — B). 

Of course, the circle is the special case of an ellipse in which b = a and the 
equation takes the form 


(xa +(y— py =a’. (1.42) 


The distinguishing characteristic of this equation is that when it is expressed in 
the form (1.38) the coefficients of x? and y are equal and that of xy is zero; this 
property is not changed by any reorientation or scaling and so acts to identify a 
general conic as a circle. 

Definitions of the conic sections in terms of geometrical properties are also 
available; for example, a parabola can be defined as the locus of a point that 
is always at the same distance from a given straight line (the directrix) as it is 
from a given point (the focus). When these properties are expressed in Cartesian 
coordinates the above equations are obtained. For a circle, the defining property 
is that all points on the curve are a distance a from (a, f); (1.42) expresses this 
requirement very directly. In the following worked example we derive the equation 
for a parabola. 


> Find the equation of a parabola that has the line x = —a as its directrix and the point 
(a,0) as its focus. 


Figure 1.3 shows the situation in Cartesian coordinates. Expressing the defining requirement 
that PN and PF are equal in length gives 
(x+a)=[(x—al ty]? = (x+aP =(x-aP ty’ 


which, on expansion of the squared terms, immediately gives y? = 4ax. This is (1.40) with 
a and f both set equal to zero. < 


Although the algebra is more complicated, the same method can be used to 
derive the equations for the ellipse and the hyperbola. In these cases the distance 
from the fixed point is a definite fraction, e, known as the eccentricity, of the 
distance from the fixed line. For an ellipse 0 < e < 1, for a circle e = 0, and for a 
hyperbola e > 1. The parabola corresponds to the case e = 1. 
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x=-a 


Figure 1.3. Construction of a parabola using the point (a,0) as the focus and 
the line x = —a as the directrix. 


The values of a and b (with a > b) in equation (1.39) for an ellipse are related 
to e through 
> @&—-P 


a 


and give the lengths of the semi-axes of the ellipse. If the ellipse is centred on 
the origin, ie. « = B = 0, then the focus is (—ae,0) and the directrix is the line 
x =—a/e. 

For each conic section curve, although we have two variables, x and y, they are 
not independent, since if one is given then the other can be determined. However, 
determining y when x is given, say, involves solving a quadratic equation on each 
occasion, and so it is convenient to have parametric representations of the curves. 
A parametric representation allows each point on a curve to be associated with 
a unique value of a single parameter t. The simplest parametric representations 
for the conic sections are as given below, though that for the hyperbola uses 
hyperbolic functions, not formally introduced until chapter 3. That they do give 
valid parameterizations can be verified by substituting them into the standard 
forms (1.39)-(1.41); in each case the standard form is reduced to an algebraic or 
trigonometric identity. 


x=at+acos¢?, y=f+bsingd (ellipse), 
x =a+at’, y=fP+2at (parabola), 
x=a+acosh¢d, y=f+bsinh¢ (hyperbola). 


As a final example illustrating several topics from this section we now prove 
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the well-known result that the angle subtended by a diameter at any point on a 
circle is a right angle. 


> Taking the diameter to be the line joining Q = (—a,0) and R = (a,0) and the point P to 
be any point on the circle x? + y* =a’, prove that angle QPR is a right angle. 


If P is the point (x,y), the slope of the line QP is 
y—0 y 


my (a) xta’ 


That of RP is 


a) ; 
Mm = y = J é 
xX—(a) x-a 
Thus 
2 
pared 
mM: = one my 
But, since P is on the circle, y? = a* — x? and consequently mjm) = —1. From result (1.24) 


this implies that QP and RP are orthogonal and that QP R is therefore a right angle. Note 
that this is true for any point P on the circle. < 


1.4 Partial fractions 


In subsequent chapters, and in particular when we come to study integration 
in chapter 2, we will need to express a function f(x) that is the ratio of two 
polynomials in a more manageable form. To remove some potential complexity 
from our discussion we will assume that all the coefficients in the polynomials 
are real, although this is not an essential simplification. 

The behaviour of f(x) is crucially determined by the location of the zeros of 
its denominator, ie. if f(x) is written as f(x) = g(x)/h(x) where both g(x) and 
h(x) are polynomials’ then f(x) changes extremely rapidly when x is close to 
those values a; that are the roots of h(x) = 0. To make such behaviour explicit, 
we write f(x) as a sum of terms such as A/(x — a)", in which A is a constant, « is 
one of the «; that satisfy h(«;) = 0 and n is a positive integer. Writing a function 
in this way is known as expressing it in partial fractions. 

Suppose, for the sake of definiteness, that we wish to express the function 


4x +2 


= x? 4+3x4+2 


S It is assumed that the ratio has been reduced so that g(x) and A(x) do not contain any common 
factors, ie. there is no value of x that makes both vanish at the same time. We may also assume 
without any loss of generality that the coefficient of the highest power of x in h(x) has been made 
equal to unity, if necessary, by dividing both numerator and denominator by the coefficient of this 
highest power. 
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in partial fractions, i.e. to write it as 


ANS g(x) 4x +2 At Ad 


hs) 2 +3x+2 omy omy (1.43) 
The first question that arises is that of how many terms there should be on 
the right-hand side (RHS). Although some complications occur when h(x) has 
repeated roots (these are considered below) it is clear that f(x) only becomes 
infinite at the two values of x, a and a, that make h(x) = 0. Consequently the 
RHS can only become infinite at the same two values of x and therefore contains 
only two partial fractions — these are the ones shown explicitly. This argument 
can be trivially extended (again temporarily ignoring the possibility of repeated 
roots of h(x)) to show that if h(x) is a polynomial of degree n then there should be 
n terms on the RHS, each containing a different root «; of the equation h(«;) = 0. 

A second general question concerns the appropriate values of the nj. This is 
answered by putting the RHS over a common denominator, which will clearly 
have to be the product (x — a )"'(x — a)” ---. Comparison of the highest power 
of x in this new RHS with the same power in h(x) shows that nj +m +--- =n. 
This result holds whether or not h(x) = 0 has repeated roots and, although we 
do not give a rigorous proof, strongly suggests the following correct conclusions. 


e The number of terms on the RHS is equal to the number of distinct roots of 
h(x) = 0, each term having a different root a; in its denominator (x — a)". 


e Ifa; is a multiple root of h(x) = 0 then the value to be assigned to n; in (1.43) is 
that of m; when h(x) is written in the product form (1.9). Further, as discussed 
on p. 23, A; has to be replaced by a polynomial of degree m; — 1. This is also 
formally true for non-repeated roots, since then both m; and n; are equal to 
unity. 


Returning to our specific example we note that the denominator h(x) has zeros 
at x = a = —1 and x = a = —2; these x-values are the simple (non-repeated) 
roots of h(x) = 0. Thus the partial fraction expansion will be of the form 


4x+2 Ar, Ap 
x2?4+3x4+2 %x+1 x42 


(1.44) 


We now list several methods available for determining the coefficients A, and 
A. We also remind the reader that, as with all the explicit examples and techniques 
described, these methods are to be considered as models for the handling of any 
ratio of polynomials, with or without characteristics that make it a special case. 


(i) The RHS can be put over a common denominator, in this case (x+1)(x+2), 
and then the coefficients of the various powers of x can be equated in the 
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(ii) 


(iii) 


numerators on both sides of the equation. This leads to 


Aye Y= Ale +2) Aste), 
4=A,+ A) 2=2A;+ Ao. 


Solving the simultaneous equations for A; and A) gives A; = —2 and 
Az = 6. 

A second method is to substitute two (or more generally n) different 
values of x into each side of (1.44) and so obtain two (or n) simultaneous 
equations for the two (or n) constants A;. To justify this practical way of 
proceeding it is necessary, strictly speaking, to appeal to method (i) above, 
which establishes that there are unique values for A; and A, valid for 
all values of x. It is normally very convenient to take zero as one of the 
values of x, but of course any set will do. Suppose in the present case that 
we use the values x = 0 and x = 1 and substitute in (1.44). The resulting 
equations are 


2 Ar Ad 
a | 2 
6 A, Ad 
6 2 
which on solution give A; = —2 and A) = 6, as before. The reader can 


easily verify that any other pair of values for x (except for a pair that 
includes a; or ) gives the same values for A, and A). 

The very reason why method (ii) fails if x is chosen as one of the roots 
a; of h(x) = 0 can be made the basis for determining the values of the A; 
corresponding to non-multiple roots without having to solve simultaneous 
equations. The method is conceptually more difficult than the other meth- 
ods presented here, and needs results from the theory of complex variables 
(chapter 24) to justify it. However, we give a practical ‘cookbook’ recipe 
for determining the coefficients. 


(a) To determine the coefficient A,, imagine the denominator h(x) 
written as the product (x — 1)(x — #2)---(x — a), with any m-fold 
repeated root giving rise to m factors in parentheses. 

(b) Now set x equal to « and evaluate the expression obtained after 
omitting the factor that reads a, — o%. 

(c) Divide the value so obtained into g(%); the result is the required 
coefficient A,. 


For our specific example we find that in step (a) that h(x) = (x + 1)(x + 2) 
and that in evaluating A; step (b) yields —1 + 2, ie. 1. Since g(—1) = 
4(—1) + 2 = —2, step (c) gives A; as (—2)/(1), Le in agreement with our 
other evaluations. In a similar way A) is evaluated as (—6)/(—1) = 6. 
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Thus any one of the methods listed above shows that 


4x+2 — —2 6 
P4axb2 x41 x42 
The best method to use in any particular circumstance will depend on the 
complexity, in terms of the degrees of the polynomials and the multiplicities of 
the roots of the denominator, of the function being considered and, to some 
extent, on the individual inclinations of the student; some prefer lengthy but 
straightforward solution of simultaneous equations, whilst others feel more at 
home carrying through shorter but more abstract calculations in their heads. 


1.4.1 Complications and special cases 


Having established the basic method for partial fractions, we now show, through 
further worked examples, how some complications are dealt with by extensions 
to the procedure. These extensions are introduced one at a time, but of course in 
any practical application more than one may be involved. 


The degree of the numerator is greater than or equal to that of the denominator 


Although we have not specifically mentioned the fact, it will be apparent from 
trying to apply method (i) of the previous subsection to such a case, that if the 
degree of the numerator (m) is not less than that of the denominator (n) then the 
ratio of two polynomials cannot be expressed in partial fractions. 

To get round this difficulty it is necessary to start by dividing the denominator 
h(x) into the numerator g(x) to obtain a further polynomial, which we will denote 
by s(x), together with a function t(x) that is a ratio of two polynomials for which 
the degree of the numerator is less than that of the denominator. The function 
t(x) can therefore be expanded in partial fractions. As a formula, 

g(x) r(x) 
PON h(x)’ 
It is apparent that the polynomial r(x) is the remainder obtained when g(x) is 
divided by h(x), and, in general, will be a polynomial of degree n — 1. It is also 
clear that the polynomial s(x) will be of degree m—n. Again, the actual division 
process can be set out as an algebraic long division sum but is probably more 
easily handled by writing (1.45) in the form 


g(x) = s(x)h(x) + r(x) (1.46) 


= s(x) + t(x) = s(x) + (1.45) 


or, more explicitly, as 


g(x) = (Sm—nx "+ Sm—n—1e rt So) h(x) + (px! + Tyox + +++ +709) 
(1.47) 


and then equating coefficients. 
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We illustrate this procedure with the following worked example. 


> Find the partial fraction decomposition of the function 
x3 4+ 3x? +2x+1 
~=-x-6 ¢ 


f(x) = 


Since the degree of the numerator is 3 and that of the denominator is 2, a preliminary 
long division is necessary. The polynomial s(x) resulting from the division will have degree 
3 —2 = 1 and the remainder r(x) will be of degree 2— 1 = 1 (or less). Thus we write 

x3 + 3x? + 2x + 1 = (91x + 5p)(x? — x — 6) + (rx + 10). 


From equating the coefficients of the various powers of x on the two sides of the equation, 
starting with the highest, we now obtain the simultaneous equations 


1=s, 

3 = 8-51, 

2 = —so — 68, +11, 
1 = —659 + ro. 


These are readily solved, in the given order, to yield s; = 1, so = 4, r; = 12 and ro = 25. 
Thus f(x) can be written as 


12x +25 

{Het 4+ oe 
The last term can now be decomposed into partial fractions as previously. The zeros of 
the denominator are at x = 3 and x = —2 and the application of any method from the 
previous subsection yields the respective constants as A; = 124 and A, = —t. Thus the 
final partial fraction decomposition of f(x) is 

Scio 61 1 ora 
5(x—3)  5(x+2) 


Factors of the form a? + x? in the denominator 


We have so far assumed that the roots of h(x) = 0, needed for the factorisation of 
the denominator of f(x), can always be found. In principle they always can but 
in some cases they are not real. Consider, for example, attempting to express in 
partial fractions a polynomial ratio whose denominator is h(x) = x? — x? +2x—2. 
Clearly x = 1 is a zero of h(x), and so a first factorisation is (x — 1)(x? + 2). 
However we cannot make any further progress because the factor x? + 2 cannot 
be expressed as (x — «)(x — f) for any real « and f. 

Complex numbers are introduced later in this book (chapter 3) and, when the 
reader has studied them, he or she may wish to justify the procedure set out 
below. It can be shown to be equivalent to that already given, but the zeros of 
h(x) are now allowed to be complex and terms that are complex conjugates of 
each other are combined to leave only real terms. 

Since quadratic factors of the form a?-+x* that appear in h(x) cannot be reduced 
to the product of two linear factors, partial fraction expansions including them 
need to have numerators in the corresponding terms that are not simply constants 
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A; but linear functions of x, 1e. of the form Bix + C;. Thus, in the expansion, 
linear terms (first-degree polynomials) in the denominator have constants (zero- 
degree polynomials) in their numerators, whilst quadratic terms (second-degree 
polynomials) in the denominator have linear terms (first-degree polynomials) in 
their numerators. As a symbolic formula, the partial fraction expansion of 


g(x) 
(x — a )(x — a) ++ (x — otp)(x? + af)(x? + a3)--- (x? + a2) 
should take the form 
A, Ad Ap Byx+C; Box + Cy Byx + Cy 
tan Se ae Ea gl ee ee 
1 xX — 2 X — Op x + ay x* + a5 x + dg 


Of course, the degree of g(x) must be less than p + 2q; if it is not, an initial 
division must be carried out as demonstrated earlier. 


Repeated factors in the denominator 


Consider trying (incorrectly) to expand 


x—4 
10) = Gre aPF 
in partial fraction form as follows: 
x-4 Ay Ay 


(x+Dx—-22 x+1° &—2)’ 

Multiplying both sides of this supposed equality by (x + 1)(x — 2)? produces an 
equation whose LHS is linear in x, whilst its RHS is quadratic. This is clearly 
wrong and so an expansion in the above form cannot be valid. The correction we 
must make is very similar to that needed in the previous subsection, namely that 
since (x — 2)° is a quadratic polynomial the numerator of the term containing it 
must be a first-degree polynomial, and not simply a constant. 

The correct form for the part of the expansion containing the doubly repeated 
root is therefore (Bx + C)/(x—2)°. Using this form and either of methods (i) and 
(ii) for determining the constants gives the full partial fraction expansion as 


x—4 _ 5 5x — 16 
(xt Dx—-22 W%K+F1 x 9(x — 2)’ 
as the reader may verify. 
Since any term of the form (Bx + C)/(x — «)* can be written as 


B(x—a)+C+ Ba B C+ Ba 


(x — a xa (x— a)’ 
and similarly for multiply repeated roots, an alternative form for the part of the 
partial fraction expansion containing a repeated root « is 

ae es ee ee Ds 
x—a |) (x—a)? - (x — at)?” 


(1.48) 


23 


PRELIMINARY ALGEBRA 


In this form, all x-dependence has disappeared from the numerators but at the 
expense of p—1 additional terms; the total number of constants to be determined 
remains unchanged, as it must. 

When describing possible methods of determining the constants in a partial 
fraction expansion, we noted that method (iii), p. 20, which avoids the need to 
solve simultaneous equations, is restricted to terms involving non-repeated roots. 
In fact, it can be applied in repeated-root situations, when the expansion is put 
in the form (1.48), but only to find the constant in the term involving the largest 
inverse power of x —a, ie. D, in (1.48). 

We conclude this section with a more protracted worked example that contains 
all three of the complications discussed. 


> Resolve the following expression F(x) into partial fractions: 


Semuy arenes oh 
ee x? — 2x4 — x° + 5x* — 46x + 100 


(x? + 6)(x — 2)? 


We note that the degree of the denominator (4) is not greater than that of the numerator 
(5), and so we must start by dividing the latter by the former. It follows, from the difference 
in degrees and the coefficients of the highest powers in each, that the result will be a linear 
expression s;x + So with the coefficient s; equal to 1. Thus the numerator of F(x) must be 
expressible as 


(x + so)(xt — 4x3 + 10x? — 24x + 24) + (3x3 + 1x? + 11x +10), 
where the second factor in parentheses is the denominator of F(x) written as a polynomial. 


Equating the coefficients of x*+ gives —2 = —4+5p and fixes sp as 2. Equating the coefficients 
of powers less than 4 gives equations involving the coefficients r; as follows: 


-1=-8+10+75, 

5 =—244+20+1, 
46 = 24-48 +1, 
100 = 48 + 10. 


Thus the remainder polynomial r(x) can be constructed and F(x) written as 


—3x3 + 9x? — 22x + 52 
(x? + 6)(x — 2)? 


F(x)=x+2+ =x+2+ f(x). 


The polynomial ratio f(x) can now be expressed in partial fraction form, noting that its 
denominator contains both a term of the form x? + a? and a repeated root. Thus 


Bx+C 
LO) aoe a ie 


We could now put the RHS of this equation over the common denominator (x? +6)(x—2) 
and find B,C,D, and D» by equating coefficients of powers of x. It is quicker, however, 
to use methods (iii) and (ii). Method (ili) gives D, as (—24 + 36 — 44 + 52)/(4+4+ 6) = 2. 
We choose to evaluate the other coefficients by method (ii), and setting x = 0, x = 1 and 
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x = —l1 gives respectively 
52. C Di % 2 
24 «6 2-4? 
36 =B+C 
> = Ate, +2, 
8 C-B D, 2 
i eg es eeecenge Oa 
63 7 3.0 «9 
These equations reduce to 
4C — 12D, = 40, 


B+C—7D, = 22, 
—9B +9C — 21D, = 72, 
with solution B = 0, C = 1, D, = —3. 
Thus, finally, we may rewrite the original expression F(x) in partial fractions as 
1 3 2 


FON SAY aang 50 Ge 


1.5 Binomial expansion 


Earlier in this chapter we were led to consider functions containing powers of 
the sum or difference of two terms, e.g. (x — a)”. Later in this book we will find 
numerous occasions on which we wish to write such a product of repeated factors 
as a polynomial in x or, more generally, as a sum of terms each of which contains 
powers of x and « separately, as opposed to a power of their sum or difference. 

To make the discussion general and the result applicable to a wide variety of 
situations, we will consider the general expansion of f(x) = (x+y)", where x and 
y may stand for constants, variables or functions and, for the time being, n is a 
positive integer. It may not be obvious what form the general expansion takes 
but some idea can be obtained by carrying out the multiplication explicitly for 
small values of n. Thus we obtain successively 


(x+y) =xt+y, 

(xt y)P = (x+y) ty) =x? + 2xy ty, 

(x+y)? = (x + yx? + 2xy + y?) = x? + 3x’y + 3xy? + y’, 

(x + yy = (x + yx? 3x7y } 3xy? f y?) =x'4 Axty - 6x7 y? + 4xy3 + yt. 


This does not establish a general formula, but the regularity of the terms in 
the expansions and the suggestion of a pattern in the coefficients indicate that a 
general formula for power n will have n+ 1 terms, that the powers of x and y in 
every term will add up to n and that the coefficients of the first and last terms 
will be unity whilst those of the second and penultimate terms will be n. 
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In fact, the general expression, the binomial expansion for power n, is given by 


k=n 
(x +4 yy" = Se Cex yk, (1.49) 
k=0 
where "Cx is called the binomial coefficient and is expressed in terms of factorial 
functions by n!/[k!(n — k)!]. Clearly, simply to make such a statement does not 
constitute proof of its validity, but, as we will see in subsection 1.5.2, (1.49) can 
be proved using a method called induction. Before turning to that proof, we 
investigate some of the elementary properties of the binomial coefficients. 


1.5.1 Binomial coefficients 


As stated above, the binomial coefficients are defined by 


n! n 
‘¢, =:__—___= forO<k<n, 1.50 

«= Tkln—b! (@) Bele (1.90) 
where in the second identity we give a common alternative notation for "Cy. 
Obvious properties include 


(i) "Co ="Cy = 1, 
(ii) "C) ="Cr1 =n, 
(iii) "Cy = "Cy. 


We note that, for any given n, the largest coefficient in the binomial expansion is 
the middle one (k = n/2) if n is even; the middle two coefficients (k = s(n + 1)) 
are equal largest if n is odd. Somewhat less obvious is the result 
n! n! 
+ 
kin—k)! (k—1)'(n—k+1)! 
— al[((nt+1—k) +k] 
k\(n+1—k)! 

n+1)! s 

= aos =™C. (1.51) 


An equivalent statement, in which k has been redefined as k + 1, is 


"C, +"Ch_4 =< 


"Ce + "Cea = "Chat. (1.52) 


1.5.2 Proof of the binomial expansion 


We are now in a position to prove the binomial expansion (1.49). In doing so, we 
introduce the reader to a procedure applicable to certain types of problems and 
known as the method of induction. The method is discussed much more fully in 
subsection 1.7.1. 
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We start by assuming that (1.49) is true for some positive integer n = N. We now 
proceed to show that this implies that it must also be true for n = N+1, as follows: 


(x+y)Nt1 = (+ SINC xN k yk 


k=0 
N N 
ax ys N Cy xNtiok yk fe we, NOyxN-k yl 
k=0 
N N+1 
ax Sea ia ae Oe (NT) Jy 
k=0 


where in the first line we have used the assumption and in the third line have 
moved the second summation index by unity, by writing k +1 = j. We now 
separate off the first term of the first sum, “Cox+!, and write it as N*!Cox%*!; 
we can do this since, as noted in (i) following (1.50), "Co = 1 for every n. Similarly, 
the last term of the second summation can be replaced by “t+!Cyy,yN*!. 

The remaining terms of each of the two summations are now written together, 


with the summation index denoted by k in both terms. Thus 


N 
(x + y)Ntt = NF eo ue es Ny, 3) x0) kyk ANH Cy pyNt 
k=1 


N 
= N+1Cy x Lae Sige kyk 4 N41Cy ry 
k= 


N+ 
- S- NATO (N DK yk 
k=0 


In going from the first to the second line we have used result (1.51). Now we 
observe that the final overall equation is just the original assumed result (1.49) 
but with n = N +1. Thus it has been shown that if the binomial expansion is 
assumed to be true for n = N, then it can be proved to be true forn = N +1. But 
it holds trivially for n = 1, and therefore for n = 2 also. By the same token it is 
valid for n = 3,4,..., and hence is established for all positive integers n. 


1.6 Properties of binomial coefficients 
1.6.1 Identities involving binomial coefficients 


There are many identities involving the binomial coefficients that can be derived 
directly from their definition, and yet more that follow from their appearance in 
the binomial expansion. Only the most elementary ones, given earlier, are worth 
committing to memory but, as illustrations, we now derive two results involving 
sums of binomial coefficients. 
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The first is a further application of the method of induction. Consider the 
proposal that, for anyn > 1 andk > 0, 


n-1 
yr sss th Ohta (1.53) 
s=0 


Notice that here n, the number of terms in the sum, is the parameter that varies, 
k is a fixed parameter, whilst s is a summation index and does not appear on the 
RHS of the equation. 

Now we suppose that the statement (1.53) about the value of the sum of the 
binomial coefficients *C,,*+!C,,...,*+"-!C, is true for n = N. We next write down 
a series with an extra term and determine the implications of the supposition for 
the new series: 

N+I-1 


N-1 
ktsc, = , ktsc, a8 KANG, 
s=0 s=0 


N+k N+k 
= SEC uy FATE C, 


= NHI GY 


But this is just proposal (1.53) with n now set equal to N + 1. To obtain the last 
line, we have used (1.52), with n set equal to N +k. 

It only remains to consider the case n = 1, when the summation only contains 
one term and (1.53) reduces to 


EG, = ad Ore 


This is trivially valid for any k since both sides are equal to unity, thus completing 
the proof of (1.53) for all positive integers n. 

The second result, which gives a formula for combining terms from two sets 
of binomial coefficients in a particular way (a kind of ‘convolution’, for readers 
who are already familiar with this term), is derived by applying the binomial 
expansion directly to the identity 


eye) Sees 


Written in terms of binomial expansions, this reads 


Pp q p+q 
: PC xP Sys : 1C,x1"y! = » pt+q CpxP tay", 
s=0 t=0 r=0 


We now equate coefficients of x’*4-"y" on the two sides of the equation, noting 
that on the LHS all combinations of s and t such that s+t =r contribute. This 
gives as an identity that 


i F 


¥ GGG =F es, (1.54) 
t=0 t=0 
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We have specifically included the second equality to emphasise the symmetrical 
nature of the relationship with respect to p and q. 

Further identities involving the coefficients can be obtained by giving x and y 
special values in the defining equation (1.49) for the expansion. If both are set 
equal to unity then we obtain (using the alternative notation so as to produce 


familiarity with it) 
n n n n P 
(+i) +G) +--+ Gi) = (1.55) 


whilst setting x = 1 and y = —1 yields 
n n n nf2\ _ 
\-\s(C)—-+rer(jen aso 


1.6.2 Negative and non-integral values of n 


Up till now we have restricted n in the binomial expansion to be a positive 
integer. Negative values can be accommodated, but only at the cost of an infinite 
series of terms rather than the finite one represented by (1.49). For reasons that 
are intuitively sensible and will be discussed in more detail in chapter 4, very 
often we require an expansion in which, at least ultimately, successive terms in 
the infinite series decrease in magnitude. For this reason, if x > y we consider 
(x +y)~"", where m itself is a positive integer, in the form 
n —m — y oe 
(x+y) =(x+yy" =x "(i+2) : 
Since the ratio y/x is less than unity, terms containing higher powers of it will be 
small in magnitude, whilst raising the unit term to any power will not affect its 
magnitude. If y > x the roles of the two must be interchanged. 
We can now state, but will not explicitly prove, the form of the binomial 
expansion appropriate to negative values of n (n equal to —m): 
ns k 
(x+y =@tym=x™D>o—ma (2), (1.57) 
k=0 . 
where the hitherto undefined quantity ~"C;, which appears to involve factorials 
of negative numbers, is given by 


,m(m+ 1)---(m+k—1) 


k—1)! ; 
7 (m+ ye (—1)k ™ 1G, 


“Cy, = (-1) —_ 
(m— Ik! (1.58) 


=(-1) 


The binomial coefficient on the extreme right of this equation has its normal 
meaning and is well defined since m+k—1>k. 

Thus we have a definition of binomial coefficients for negative integer values 
of n in terms of those for positive n. The connection between the two may not 
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be obvious, but they are both formed in the same way in terms of recurrence 
relations. Whatever the sign of n, the series of coefficients "C, can be generated 
by starting with "Co = 1 and using the recurrence relation 


ae (1.59) 


The difference is that for positive integer n the series terminates when k = n, 
whereas for negative n there is no such termination — in line with the infinite 
series of terms in the corresponding expansion. 

Finally we note that, in fact, equation (1.59) generates the appropriate coef- 
ficients for all values of n, positive or negative, integer or non-integer, with the 
obvious exception of the case in which x = —y and n is negative. For non-integer 
n the expansion does not terminate, even if n is positive. 


1.7 Some particular methods of proof 


Much of the mathematics used by physicists and engineers is concerned with 
obtaining a particular value, formula or function from a given set of data and 
stated conditions. However, just as it is essential in physics to formulate the basic 
laws and so be able to set boundaries on what can or cannot happen, so it 
is important in mathematics to be able to state general propositions about the 
outcomes that are or are not possible. To this end one attempts to establish 
theorems that state in as general a way as possible mathematical results that 
apply to particular types of situation. We conclude this introductory chapter by 
describing two methods that can sometimes be used to prove particular classes 
of theorems. 

The two general methods of proof are known as proof by induction (which 
has already been met in this chapter) and proof by contradiction. They share 
the common characteristic that at an early stage in the proof an assumption 
is made that a particular (unproven) statement is true; the consequences of 
that assumption are then explored. In an inductive proof the conclusion is 
reached that the assumption is self-consistent and has other equally consistent 
but broader implications, which are then applied to establish the general validity 
of the assumption. A proof by contradiction, however, establishes an internal 
inconsistency and thus shows that the assumption is unsustainable; the natural 
consequence of this is that the negative of the assumption is established as true. 

Later in this book use will be made of these methods of proof to explore new 
territory, e.g. to examine the properties of vector spaces, matrices and groups. 
However, at this stage we will draw our illustrative and test examples from earlier 
sections of this chapter and other topics in elementary algebra and number theory. 
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1.7.1 Proof by induction 


The proof of the binomial expansion given in subsection 1.5.2 and the identity 
established in subsection 1.6.1 have already shown the way in which an inductive 
proof is carried through. They also indicated the main limitation of the method, 
namely that only an initially supposed result can be proved. Thus the method 
of induction is of no use for deducing a previously unknown result; a putative 
equation or result has to be arrived at by some other means, usually by noticing 
patterns or by trial and error using simple values of the variables involved. It 
will also be clear that propositions that can be proved by induction are limited 
to those containing a parameter that takes a range of integer values (usually 
infinite). 

For a proposition involving a parameter n, the five steps in a proof using 
induction are as follows. 


(i) Formulate the supposed result for general n. 

(ii) Suppose (i) to be true for n = N (or more generally for all values of 
n < N; see below), where N is restricted to lie in the stated range. 

(iii) Show, using only proven results and supposition (ii), that proposition (i) 
is true forn = N +1. 

(iv) Demonstrate directly, and without any assumptions, that proposition (1) is 

true when n takes the lowest value in its range. 

It then follows from (iii) and (iv) that the proposition is valid for all values 

of n in the stated range. 


~— 


(v 


(It should be noted that, although many proofs at stage (iii) require the validity 
of the proposition only for n = N, some require it for all n less than or equal to N 
— hence the form of inequality given in parentheses in the stage (11) assumption.) 

To illustrate further the method of induction, we now apply it to two worked 
examples; the first concerns the sum of the squares of the first n natural numbers. 


> Prove that the sum of the squares of the first n natural numbers is given by 


Sor? = §n(n + 1)(2n+ 1). 
r=1 


As previously we start by assuming the result is true for n = N. Then it follows that 


N+1 N 
Sirs r+(N+1y 
r=1 r=1 
= £N(N + 1)(2N +1) +(N +1) 
= 7(N + 1)[NQN + 1) + 6N + 6] 
= 2(N + 1)[(2N + 3)(N + 2)] 
= 2(N +1 [(N +1) + 12(N +1) 4-1. 
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This is precisely the original assumption, but with N replaced by N + 1. To complete the 
proof we only have to verify (1.60) for n = 1. This is trivially done and establishes the 
result for all positive n. The same and related results are obtained by a different method 
in subsection 4.2.5. < 


Our second example is somewhat more complex and involves two nested proofs 
by induction: whilst trying to establish the main result by induction, we find that 
we are faced with a second proposition which itself requires an inductive proof. 


> Show that Q(n) = n* + 2n? + 2n? +n is divisible by 6 (without remainder ) for all positive 


integer values of n. 


Again we start by assuming the result is true for some particular value N of n, whilst 
noting that it is trivially true for n = 0. We next examine Q(N + 1), writing each of its 
terms as a binomial expansion: 
O(N +1) =(N+1)°4+2N 41° +2(.N +1? +(N +1) 
= (N*+4N?+6N? +4N + 1)+2(N? +3N?43N +1) 
+2(N? +2N + 1)+(N +1) 

= (N*4 + 2N? +. 2N? +N) +(4N? + 12N? + 14N + 6). 
Now, by our assumption, the group of terms within the first parentheses in the last line 
is divisible by 6 and clearly so are the terms 12N? and 6 within the second parentheses. 
Thus it comes down to deciding whether 4N? + 14N is divisible by 6 — or equivalently, 
whether R(N) = 2N? +7N is divisible by 3. 

To settle this latter question we try using a second inductive proof and assume that 
R(N) is divisible by 3 for N = M, whilst again noting that the proposition is trivially true 
for N = M = 0. This time we examine R(M + 1): 

R(M + 1) =2(M +1)? +7(M +1) 
= 2(M? +3M?+3M+1)+7(M +1) 
= (2M? +7M) 4+ 3(2M? + 2M +3) 
By assumption, the first group of terms in the last line is divisible by 3 and the second 
group is patently so. We thus conclude that R(N) is divisible by 3 for all N > M, and 
taking M = 0 shows that it is divisible by 3 for all N. 

We can now return to the main proposition and conclude that since R(N) = 2N* + 7N 

is divisible by 3, 4N? + 12N? + 14N +6 is divisible by 6. This in turn establishes that the 


divisibility of Q(N + 1) by 6 follows from the assumption that Q(N) divides by 6. Since 
Q(0) clearly divides by 6, the proposition in the question is established for all values of n. < 


1.7.2 Proof by contradiction 


The second general line of proof, but again one that is normally only useful when 
the result is already suspected, is proof by contradiction. The questions it can 
attempt to answer are only those that can be expressed in a proposition that 
is either true or false. Clearly, it could be argued that any mathematical result 
can be so expressed but, if the proposition is no more than a guess, the chances 
of success are negligible. Valid propositions containing even modest formulae 
are either the result of true inspiration or, much more normally, yet another 
reworking of an old chestnut! 
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The essence of the method is to exploit the fact that mathematics is required 
to be self-consistent, so that, for example, two calculations of the same quantity, 
starting from the same given data but proceeding by different methods, must give 
the same answer. Equally, it must not be possible to follow a line of reasoning and 
draw a conclusion that contradicts either the input data or any other conclusion 
based upon the same data. 

It is this requirement on which the method of proof by contradiction is based. 
The crux of the method is to assume that the proposition to be proved is 
not true, and then use this incorrect assumption and ‘watertight’ reasoning to 
draw a conclusion that contradicts the assumption. The only way out of the 
self-contradiction is then to conclude that the assumption was indeed false and 
therefore that the proposition is true. 

It must be emphasised that once a (false) contrary assumption has been made, 
every subsequent conclusion in the argument must follow of necessity. Proof by 
contradiction fails if at any stage we have to admit ‘this may or may not be 
the case’. That is, each step in the argument must be a necessary consequence of 
results that precede it (taken together with the assumption), rather than simply a 
possible consequence. 

It should also be added that if no contradiction can be found using sound 
reasoning based on the assumption then no conclusion can be drawn about either 
the proposition or its negative and some other approach must be tried. 

We illustrate the general method with an example in which the mathematical 
reasoning is straightforward, so that attention can be focussed on the structure 
of the proof. 


>A rational number r is a fraction r = p/q in which p and q are integers with q positive. 
Further, r is expressed in its lowest terms, any integer common factor of p and q having 


been divided out. 
Prove that the square root of an integer m cannot be a rational number, unless the square 
root itself is an integer. 


We begin by supposing that the stated result is not true and that we can write an equation 


Jm=r= 7 for integers m,p,q with q #1. 


It then follows that p? = mq?. But, since r is expressed in its lowest terms, p and q, and 
hence p” and q’, have no factors in common. However, m is an integer; this is only possible 
if q = 1 and p* =m. This conclusion contradicts the requirement that q # 1 and so leads 
to the conclusion that it was wrong to suppose that a/m can be expressed as a non-integer 
rational number. This completes the proof of the statement in the question. < 


Our second worked example, also taken from elementary number theory, 
involves slightly more complicated mathematical reasoning but again exhibits the 
structure associated with this type of proof. 
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>The prime integers p; are labelled in ascending order, thus p; = 1, pr = 2, ps = 7, etc. 
Show that there is no largest prime number. 


Assume, on the contrary, that there is a largest prime and let it be py. Consider now the 
number q formed by multiplying together all the primes from p,; to py and then adding 
one to the product, i.e. 


q=Pip2:: pn +1. 


By our assumption py is the largest prime, and so no number can have a prime factor 
greater than this. However, for every prime p;, i = 1,2,...,N, the quotient q/p; has the 
form M; +(1/p;) with M; an integer and 1/p; non-integer. This means that q/p; cannot be 
an integer and so p; cannot be a divisor of q. 

Since q is not divisible by any of the (assumed) finite set of primes, it must be itself a 
prime. As q is also clearly greater than py, we have a contradiction. This shows that our 
assumption that there is a largest prime integer must be false, and so it follows that there 
is no largest prime integer. 

It should be noted that the given construction for q does not generate all the primes 
that actually exist (e.g. for N = 3,q = 7 rather than the next actual prime value of 5, is 
found), but this does not matter for the purposes of our proof by contradiction. < 


1.7.3 Necessary and sufficient conditions 


As the final topic in this introductory chapter, we consider briefly the notion 
of, and distinction between, necessary and sufficient conditions in the context 
of proving a mathematical proposition. In ordinary English the distinction is 
well defined, and that distinction is maintained in mathematics. However, in 
the authors’ experience students tend to overlook it and assume (wrongly) that, 
having proved that the validity of proposition A implies the truth of proposition 
B, it follows by ‘reversing the argument’ that the validity of B automatically 
implies that of A. 

As an example, let proposition A be that an integer N is divisible without 
remainder by 6, and proposition B be that N is divisible without remainder by 
2. Clearly, if A is true then it follows that B is true, i.e. A is a sufficient condition 
for B; it is not however a necessary condition, as is trivially shown by taking N 
as 8. Conversely, the same value of N shows that whilst the validity of B is a 
necessary condition for A to hold, it is not sufficient. 

An alternative terminology to ‘necessary’ and ‘sufficient’ often employed by 
mathematicians is that of ‘if? and ‘only if’, particularly in the combination ‘if and 
only if? which is usually written as IFF or denoted by a double-headed arrow 
<= . The equivalent statements can be summarised by 


Aif B A is true if B is true or B= A, 
B is a sufficient condition for A B= A, 
A only if B_ A is true only if B is true or A==>B, 


B is a necessary consequence of A A ==> B, 
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AIFF B_ Ais true if and only if B is true or B<> A, 
A and B necessarily imply each other B <=> A. 


Although at this stage in the book we are able to employ for illustrative purposes 
only simple and fairly obvious results, the following example is given as a model 
of how necessary and sufficient conditions should be proved. The essential point 
is that for the second part of the proof (whether it be the ‘necessary’ part or the 
‘sufficient’ part) one needs to start again from scratch; more often than not, the 
lines of the second part of the proof will not be simply those of the first written 
in reverse order. 


> Prove that (A) a function f(x) is a quadratic polynomial with zeros at x = 2 and x = 3 
if and only if (B) the function f(x) has the form A(x? —5x +6) with 4 a non-zero constant. 


(1) Assume A, ie. that f(x) is a quadratic polynomial with zeros at x = 2 and x = 3. Let 
its form be ax? + bx +c with a # 0. Then we have 


4a+2b+c=0, 
9a+3b+c=0, 


and subtraction shows that 5a+b = 0 and b = —Sa. Substitution of this into the first of 
the above equations gives c 4a —2b 4a + 10a = 6a. Thus, it follows that 


f(x) = a(x? — 5x +6) with a#0, 
and establishes the ‘A only if B’ part of the stated result. 


(2) Now assume that f(x) has the form A(x? — 5x + 6) with 2 a non-zero constant. Firstly 
we note that f(x) is a quadratic polynomial, and so it only remains to prove that its 
zeros occur at x = 2 and x = 3. Consider f(x) = 0, which, after dividing through by the 
non-zero constant 4, gives 


x? —5x+6=0. 
We proceed by using a technique known as completing the square, for the purposes of 


illustration, although the factorisation of the above equation should be clear to the reader. 
Thus we write 


x —5x+(3P —(3Y +6 =0, 


$2 _ 4 
(x — 5) = 


The two roots of f(x) = 0 are therefore x = 2 and x = 3; these x-values give the zeros 
of f(x). This establishes the second (‘A if B’) part of the result. Thus we have shown 
that the assumption of either condition implies the validity of the other and the proof is 
complete. < 


It should be noted that the propositions have to be carefully and precisely 
formulated. If, for example, the word ‘quadratic’ were omitted from A, statement 
B would still be a sufficient condition for A but not a necessary one, since f(x) 
could then be x3 —4x?+x+6 and A would not require B. Omitting the constant 
2 from the stated form of f(x) in B has the same effect. Conversely, if A were to 
state that f(x) = 3(x —2)(x — 3) then B would be a necessary condition for A but 
not a sufficient one. 
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1.1 


12 


1.3 


1.4 


15 


1.6 


1.7 


1.8 Exercises 


Polynomial equations 


Continue the investigation of equation (1.7), namely 
g(x) = 4x3 + 3x? — 6x — 1, 
as follows. 


(a) Make a table of values of g(x) for integer values of x between —2 and 2. Use 
it and the information derived in the text to draw a graph and so determine 
the roots of g(x) = 0 as accurately as possible. 

(b) Find one accurate root of g(x) = 0 by inspection and hence determine precise 
values for the other two roots. 

(c) Show that f(x) = 4x3 + 3x? — 6x —k = 0 has only one real root unless 
—5<k<}. 


Determine how the number of real roots of the equation 
g(x) = 4x3 — 17x? + 10x +k =0 


depends upon k. Are there any cases for which the equation has exactly two 
distinct real roots? 
Continue the analysis of the polynomial equation 


f(x) =x" + 5x® + x43 +x? -2=0, 
investigated in subsection 1.1.1, as follows. 


(a) By writing the fifth-degree polynomial appearing in the expression for f’(x) 
in the form 7x° + 30x* + a(x — b)* +c, show that there is in fact only one 
positive root of f(x) =0. 

(b) By evaluating f(1), f(0) and f(—1), and by inspecting the form of f(x) for 
negative values of x, determine what you can about the positions of the real 
roots of f(x) = 0. 


Given that x = 2 is one root of 
g(x) = 2x* + 4x3 — 9x? — 11x —6 =0, 


use factorisation to determine how many real roots it has. 

Construct the quadratic equations that have the following pairs of roots: 

(a) —6, —3; (b) 0,4; (c) 2,2; (d) 3 + 2i,3 — 2i, where ? = —1. 

Use the results of (i) equation (1.13), (ii) equation (1.12) and (iii) equation (1.14) 
to prove that if the roots of 3x* — x? — 10x +8 = 0 are a, and «3 then 


(a) op! +o5! +05! = 5/4, 

os + 05 + of = 61/9, 

of + 03 + 03 = —125/27. 

Convince yourself that eliminating (say) «2 and «3 from (i), (ii) and (iii) does 
not give a simple explicit way of finding «. 


a ds 
Bese 


Trigonometric identities 
Prove that 


by considering 
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1.8 


1.9 


1.10 


1.13 


(a) the sum of the sines of 2/3 and 7/6, 
(b) the sine of the sum of 2/3 and 7/4. 


The following exercises are based on the half-angle formulae. 


(a) Use the fact that sin(z/6) = 1/2 to prove that tan(z/12) = 2— 3. 
(b) Use the result of (a) to show further that tan(z/24) = q(2 — q) where 
g=24+ 3. 


Find the real solutions of 


(a) 3sin0 —4cos@ = 2, 
(b) 4sin0 + 3cos@ = 6, 
(c) 12sin0 —5cosé =—6. 
If s = sin(z/8), prove that 
8st — 857 +1=0, 

and hence show that s = [(2— /2)/4]!/?. 
Find all the solutions of 

sin @ + sin 40 = sin 20 + sin 30 


that lie in the range —z < 0 < nm. What is the multiplicity of the solution 6 = 0? 


Coordinate geometry 

Obtain in the form (1.38) the equations that describe the following: 

(a) a circle of radius 5 with its centre at (1,—1); 

(b) the line 2x + 3y +4 =0 and the line orthogonal to it which passes through 
(1,0); 

(c) an ellipse of eccentricity 0.6 with centre (1,1) and its major axis of length 10 
parallel to the y-axis. 

Determine the forms of the conic sections described by the following equations: 

(a) x*+y?+ 6x + 8y =0; 

(b) 9x? — 4y? — 54x — 16y + 29 =0; 

(c) 2x? 4+2y?+5xy—4x+y—6=0; 

(d) x? + y?+ 2xy —8x4+ 8y =0. 


For the ellipse 


with eccentricity e, the two points (—ae,0) and (ae,0) are known as its foci. Show 
that the sum of the distances from any point on the ellipse to the foci is 2a. (The 
constancy of the sum of the distances from two fixed points can be used as an 
alternative defining property of an ellipse.) 


Partial fractions 
Resolve the following into partial fractions using the three methods given in 
section 1.4, verifying that the same decomposition is obtained by each method: 
2x+1 4 
a) ——.——_ b 
(a) 24 3x—10° (b) 


x? — 3x 
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1.16 


1.18 


1.19 


1.20 


1.21 


1:22 


1.23 
1.24 


125 


1.26 


Express the following in partial fraction form: 


2x3 —5x+1 xr+x-—1 
¢ — b) —————.. 
(a) x? — 2x — 8’ i ie rr 
Rearrange the following functions in partial fraction form: 
x-6 x3 4+ 3x? +x+19 
(a) (b) 


37x? +4x-—4’ x44+10x74+9 
Resolve the following into partial fractions in such a way that x does not appear 
in any numerator: 

2x7+x+1 x? —2 w-x-1 


OGotpessy | ‘'Sasesin “ GEDasD 


Binomial expansion 


Evaluate those of the following that are defined: (a) >C3, (b) 3Cs, (c) C3, (d) 
3 

Cs. 
Use a binomial expansion to evaluate 1/,/4.2 to five places of decimals, and 
compare it with the accurate answer obtained using a calculator. 


Proof by induction and contradiction 
Prove by induction that 


s r= n(n + 1) and a P= qir(n +1). 
r=1 r=1 


Prove by induction that 


i n+l 
Lert qe trek pee prt = 


1-r—- 


Prove that 37" +7, where n is a non-negative integer, is divisible by 8. 
If a sequence of terms, u,, satisfies the recurrence relation u,,, = (1 — x)u, + nx, 
with u; = 0, show, by induction, that, for n> 1, 
1 
n= —[nx —1+(1—x)"]. 
x 
Prove by induction that 


So ag = ae ae —cotd 
= 2r or = Qn Qn 


The quantities a; in this exercise are all positive real numbers. 


(a) Show that 
2 
a, +a 
< | —— : 
ie ( 2 ) 


(b) Hence prove, by induction on m, that 
ater te)! 
a= > oe 


where p = 2” with m a positive integer. Note that each increase of m by 
unity doubles the number of factors in the product. 


ayans-ay < ( 
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1.27 


1.28 


1.29 


1.30 


1.32 


1.33 


1.1 


1.3 


1.5 
1.7 
1.9 


Establish the values of k for which the binomial coefficient ’C; is divisible by p 
when p is a prime number. Use your result and the method of induction to prove 
that n? — n is divisible by p for all integers n and all prime numbers p. Deduce 
that n° — n is divisible by 30 for any integer n. 

An arithmetic progression of integers a, is one in which a, = dg + nd, where do 
and d are integers and n takes successive values 0, 1,2,.... 


(a) Show that if any one term of the progression is the cube of an integer then 
so are infinitely many others. 

(b) Show that no cube of an integer can be expressed as 7n +5 for some positive 
integer n. 


Prove, by the method of contradiction, that the equation 
x" $b dy 1x") $+ + a,x + ao =0, 


in which all the coefficients a; are integers, cannot have a rational root, unless 
that root is an integer. Deduce that any integral root must be a divisor of ay and 
hence find all rational roots of 


(a) x*+ 6x3 + 4x? +5x+4=0, 
(b) x4 + 5x3 + 2x? — 10x + 6 = 0. 


Necessary and sufficient conditions 


Prove that the equation ax? + bx +c = 0, in which a, b and c are real and a > 0, 
has two real distinct solutions IFF b? > 4ac. 

For the real variable x, show that a sufficient, but not necessary, condition for 
f(x) = x(x + 1)(2x + 1) to be divisible by 6 is that x is an integer. 

Given that at least one of a and b, and at least one of c and d, are non-zero, 
show that ad = bc is both a necessary and sufficient condition for the equations 


ax + by =0, 
cx +dy =0, 


to have a solution in which at least one of x and y is non-zero. 

The coefficients a; in the polynomial Q(x) = ayx* + a3x* + ax? + aix are all 
integers. Show that Q(n) is divisible by 24 for all integers n > 0 if and only if all 
of the following conditions are satisfied: 

(i) 2a4 + ay is divisible by 4; 

(ii) a4 + a2 is divisible by 12; 

(iii) ag + a3 + a2 + ay Is divisible by 24. 


1.9 Hints and answers 


(b) The roots are 1, #(—7 + /33) 0.1569, +(—7 — ,/33) = —1.593. (c) —5 and 


i are the values of k that make f(—1) and f(4) equal to zero. 


(a) a=4,b= : andc= 3 are all positive. Therefore f’(x) > 0 for all x > 0. 

(b) fC) =5, f(0) 2 and f(—1) = 5, and so there is at least one root in each 
of the ranges 0 < x < 1 and —1 < x <0. (x7 +. 5x°) + (x* — x3) + (x? — 2) 
is positive definite for —5 < x < —,/2. There are therefore no roots in this 
range, but there must be one to the left of x = —5S. 

(a) x7 +9x + 18 = 0; (b) x? —4x =0; (c) x? —4x +4 = 0; (d) x? —6x +13 =0. 

(a) Use sin(z/4) = 1/,/2. (b) Use results (1.20) and (1.21). 

(a) 1.339, —2.626. (b) No solution because 67 > 4? + 37. (c) —0.0849, —2.276. 
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1.13 


1.33 


Show that the equation is equivalent to sin(50/2) sin(0) sin(@/2) = 0. 
Solutions are —42/5,—27/5,0,27/5,42/5,2. The solution 0 = 0 has multiplicity 
3. 


(a) A circle of radius 5 centred on (—3, —4). 
(b) A hyperbola with ‘centre’ (3,—2) and ‘semi-axes’ 2 and 3. 
(c) The expression factorises into two lines, x + 2y —3 = 0 and 2x+y+2=0. 
(d) Write the expression as (x+y)? = 8(x—y) to see that it represents a parabola 
passing through the origin, with the line x + y = 0 as its axis of symmetry. 
(a) + (b) ++" 
W(x—2)  7(x+5)’ 3x  3(x— 3) 
x+2 1 x+1 2 
a) ——— —- — b) ——- + ——.. 
seer x-1 Nao eed 


(a) 10, (b) not defined, (c) —35, (d) —21. 

Look for factors common to the n = N sum and the additional n = N + 1 term, 
so as to reduce the sum for n = N +1 to a single term. 

Write 37" as 8m — 7. 

Use the half-angle formulae of equations (1.32) to (1.34) to relate functions of 
0/2* to those of 0/2'*!. 

Divisible for k = 1,2,...,p —1. Expand (n+ 1)? as n? + soe PCynk + 1. Apply 
the stated result for p = 5. Note that n> —n = n(n—1)(n4+ 1)(n? +1); the product 
of any three consecutive integers must divide by both 2 and 3. 

By assuming x = p/q with q # 1, show that a fraction —p"/q is equal to an 
integer dnp"! ++--+aipq"? + aoq""!. This is a contradiction, and is only 
resolved if g = 1 and the root is an integer. 

(a) The only possible candidates are +1,+2,+4. None is a root. 

(b) The only possible candidates are +1,+2,+3,+6. Only —3 is a root. 

f(x) can be written as x(x + 1)(x + 2) + x(x + 1)(x — 1). Each term consists of 
the product of three consecutive integers, of which one must therefore divide by 
2 and (a different) one by 3. Thus each term separately divides by 6, and so 
therefore does f(x). Note that if x is the root of 2x? + 3x* + x — 24 = 0 that lies 
near the non-integer value x = 1.826, then x(x + 1)(2x + 1) = 24 and therefore 
divides by 6. 

Note that, e.g., the condition for 6a4 + a3 to be divisible by 4 is the same as the 
condition for 2a4 + a3 to be divisible by 4. 

For the necessary (only if) part of the proof set n = 1,2,3 and take integer 
combinations of the resulting equations. 

For the sufficient (if) part of the proof use the stated conditions to prove the 
proposition by induction. Note that n° — n is divisible by 6 and that n? +n is 
even. 


a 


ot 


e 
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Preliminary calculus 


This chapter is concerned with the formalism of probably the most widely used 
mathematical technique in the physical sciences, namely the calculus. The chapter 
divides into two sections. The first deals with the process of differentiation and the 
second with its inverse process, integration. The material covered is essential for 
the remainder of the book and serves as a reference. Readers who have previously 
studied these topics should ensure familiarity by looking at the worked examples 
in the main text and by attempting the exercises at the end of the chapter. 


2.1 Differentiation 


Differentiation is the process of determining how quickly or slowly a function 
varies, as the quantity on which it depends, its argument, is changed. More 
specifically it is the procedure for obtaining an expression (numerical or algebraic) 
for the rate of change of the function with respect to its argument. Familiar 
examples of rates of change include acceleration (the rate of change of velocity) 
and chemical reaction rate (the rate of change of chemical composition). Both 
acceleration and reaction rate give a measure of the change of a quantity with 
respect to time. However, differentiation may also be applied to changes with 
respect to other quantities, for example the change in pressure with respect to a 
change in temperature. 

Although it will not be apparent from what we have said so far, differentiation 
is in fact a limiting process, that is, it deals only with the infinitesimal change in 
one quantity resulting from an infinitesimal change in another. 


2.1.1 Differentiation from first principles 


Let us consider a function f(x) that depends on only one variable x, together with 
numerical constants, for example, f(x) = 3x? or f(x) = sinx or f(x) = 24+ 3/x. 
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) 


| 
x x+Ax 


Figure 2.1 The graph of a function f(x) showing that the gradient or slope 
of the function at P, given by tan 0, is approximately equal to Af /Ax. 


Figure 2.1 shows an example of such a function. Near any particular point, 
P, the value of the function changes by an amount Af, say, as x changes 
by a small amount Ax. The slope of the tangent to the graph of f(x) at P 
is then approximately Af/Ax, and the change in the value of the function is 
Af = f(x + Ax) — f(x). In order to calculate the true value of the gradient, or 
first derivative, of the function at P, we must let Ax become infinitesimally small. 
We therefore define the first derivative of f(x) as 

df (x) f(x + Ax) — f(x) 


LOE ey Ax 


(2.1) 


provided that the limit exists. The limit will depend in almost all cases on the 
value of x. If the limit does exist at a point x =a then the function is said to be 
differentiable at a; otherwise it is said to be non-differentiable at a. The formal 
concept of a limit and its existence or non-existence is discussed in chapter 4; for 
present purposes we will adopt an intuitive approach. 

In the definition (2.1), we allow Ax to tend to zero from either positive or 
negative values and require the same limit to be obtained in both cases. A 
function that is differentiable at a is necessarily continuous at a (there must be 
no jump in the value of the function at a), though the converse is not necessarily 
true. This latter assertion is illustrated in figure 2.1: the function is continuous 
at the ‘kink’ A but the two limits of the gradient as Ax tends to zero from 
positive or negative values are different and so the function is not differentiable 
at A. 

It should be clear from the above discussion that near the point P we may 


42 


2.1 DIFFERENTIATION 


approximate the change in the value of the function, Af, that results from a small 
change Ax in x by 


df (x) 


dx 


Af = 


Ax. (2.2) 


As one would expect, the approximation improves as the value of Ax is reduced. 
In the limit in which the change Ax becomes infinitesimally small, we denote it 
by the differential dx, and (2.2) reads 


df = Way (2.3) 


This equality relates the infinitesimal change in the function, df, to the infinitesimal 
change dx that causes it. 

So far we have discussed only the first derivative of a function. However, we 
can also define the second derivative as the gradient of the gradient of a function. 
Again we use the definition (2.1) but now with f(x) replaced by f’(x). Hence the 
second derivative is defined by 


f(x + Ax) = f'(x) 
Ax : 


f"x) = Jim (2.4) 


provided that the limit exists. A physical example of a second derivative is the 
second derivative of the distance travelled by a particle with respect to time. Since 
the first derivative of distance travelled gives the particle’s velocity, the second 
derivative gives its acceleration. 

We can continue in this manner, the nth derivative of the function f(x) being 
defined by 


SOM e+ Ax) = FM) 
Ax ; 


f(x) = Jim, (2.5) 


It should be noted that with this notation f’(x) = f(x), f"(x) = f(x), etc., and 
that formally f(x) = f(x). 

All this should be familiar to the reader, though perhaps not with such formal 
definitions. The following example shows the differentiation of f(x) = x? from first 
principles. In practice, however, it is desirable simply to remember the derivatives 
of standard functions; the techniques given in the remainder of this section can 
be applied to find more complicated derivatives. 
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> Find from first principles the derivative with respect to x of f(x) = x’. 


Using the definition (2.1), 


ee ae Ax 
(x +Axy — x? 

lim 
Ax>0 Ax 

= Ij 2xAx + (Ax) 

~ AK0 Ax 

= lim (2x + Ax). 
Ax>0 


As Ax tends to zero, 2x + Ax tends towards 2x, hence 


f'(x) = 2x. < 


Derivatives of other functions can be obtained in the same way. The derivatives 
of some simple functions are listed below (note that a is a constant): 


d d d 1 
ae (x") = nx"}, Te (e**) = ae™, ae (Inax) = -, 
x x dx x 
d. d : d 
— (sinax) = acosax, —(cosax) =—asinax, — (secax) = asecax tanax, 
dx dx dx 
d d 
oe (tan ax) = asec? ax, a (cosec ax) = —a cosec ax cot ax, 
x dx 
d > ds. 1x 1 
— (cot ax) = —a cosec* ax, —_ (sin ~) = ———., 
dx dx a Q. — x2 


d 1X —1 d 1% a 
cos = tan = ; 
re a) eee ral i) a2 + x? 
Differentiation from first principles emphasises the definition of a derivative as 
the gradient of a function. However, for most practical purposes, returning to the 
definition (2.1) is ttme consuming and does not aid our understanding. Instead, as 
mentioned above, we employ a number of techniques, which use the derivatives 
listed above as ‘building blocks’, to evaluate the derivatives of more complicated 
functions than hitherto encountered. Subsections 2.1.2—2.1.7 develop the methods 
required. 


2.1.2 Differentiation of products 


As a first example of the differentiation of a more complicated function, we 
consider finding the derivative of a function f(x) that can be written as the 
product of two other functions of x, namely f(x) = u(x)v(x). For example, if 
f(x) = x'sinx then we might take u(x) = x? and v(x) = sinx. Clearly the 


44 


2.1 DIFFERENTIATION 


separation is not unique. (In the given example, possible alternative break-ups 
would be u(x) = x’, v(x) = xsinx, or even u(x) = x* tan x, v(x) = x7! cos x.) 

The purpose of the separation is to split the function into two (or more) parts, 
of which we know the derivatives (or at least we can evaluate these derivatives 
more easily than that of the whole). We would gain little, however, if we did 
not know the relationship between the derivative of f and those of u and v. 
Fortunately, they are very simply related, as we shall now show. 

Since f(x) is written as the product u(x)v(x), it follows that 


f(x + Ax) — f(x) = u(x + Ax)vo(x + Ax) — u(x)v(x) 
= u(x + Ax) [v(x + Ax) — v(x)] + [u(x + Ax) — u(x)]v(x). 


From the definition of a derivative (2.1), 


df jim {8 +Ax)— SO) 


dx — Ax30 Ax 


ee v(x + Ax) — v(x) u(x + Ax) — u(x) ; 
= jim, {us + Ax) | Kee 4 ‘Aye v(x) >. 
In the limit Ax — 0, the factors in square brackets become dv/dx and du/dx 


(by the definitions of these quantities) and u(x + Ax) simply becomes u(x). 
Consequently we obtain 


df d d d 
af = F[ulxpo(x)] = w(x) oe) + WD (x) (2.6) 


In primed notation and without writing the argument x explicitly, (2.6) is stated 
concisely as 


f’ =(w) =w' +’. (2.7) 


This is a general result obtained without making any assumptions about the 
specific forms f, u and v, other than that f(x) = u(x)v(x). In words, the result 
reads as follows. The derivative of the product of two functions is equal to the 
first function times the derivative of the second plus the second function times the 
derivative of the first. 


> Find the derivative with respect to x of f(x) = x} sinx. 


Using the product rule, (2.6), 


d_ 3. 34. a 
7g sinx) =x 7x (sin. x) + F(x ) sin x 


= x>cosx + 3x’ sinx. < 


The product rule may readily be extended to the product of three or more 
functions. Considering the function 


F(x) = u(x)o(x)w(x) (2.8) 
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and using (2.6), we obtain, as before omitting the argument, 


df d du 
oe (vw) + 7a 
Using (2.6) again to expand the first term on the RHS gives the complete result 
d dw dv du 
— = w— +u— — 2.9 
me sae dx Wee! ss dx.” G2) 
or 
(uvw) = uvw' + uv'w + u'ow. (2.10) 


It is readily apparent that this can be extended to products containing any number 
n of factors; the expression for the derivative will then consist of n terms with 
the prime appearing in successive terms on each of the n factors in turn. This is 
probably the easiest way to recall the product rule. 


2.1.3 The chain rule 


Products are just one type of complicated function that we may encounter in 
differentiation. Another is the function of a function, e.g. f(x) = (3 +x)? = u(x), 
where u(x) = 3+ x?. If Af, Au and Ax are small finite quantities, it follows that 


Af — Af Au. 
Ax Au Ax’ 
As the quantities become infinitesimally small we obtain 
df — df du 
ae > dade (2.11) 


This is the chain rule, which we must apply when differentiating a function of a 
function. 


> Find the derivative with respect to x of f(x) =(3+x?). 


Rewriting the function as f(x) = u3, where u(x) = 3 + x’, and applying (2.11) we find 
df 


dx x 


=3u £6 +x?) = 3’ x 2x = 6x(3 +x’). < 


Similarly, the derivative with respect to x of f(x) = 1/v(x) may be obtained by 
rewriting the function as f(x) = v~! and applying (2.11): 
df _ dv 1 dv 
=p? =, 2.12 
dx arr v? dx CY) 
The chain rule is also useful for calculating the derivative of a function f with 
respect to x when both x and f are written in terms of a variable (or parameter), 
say t. 


46 


2.1 DIFFERENTIATION 


> Find the derivative with respect to x of f(t) = 2at, where x = at?. 


We could of course substitute for ¢ and then differentiate f as a function of x, but in this 
case it is quicker to use 


dx dtdx "at 
where we have used the fact that 
dt _(ax\” 
dx \dt : 


df _ df dt 1 ot 
t 


2.1.4 Differentiation of quotients 


Applying (2.6) for the derivative of a product to a function f(x) = u(x)[1/v(x)], 
we may obtain the derivative of the quotient of two factors. Thus 


r ' 1\' ,f1 ' f 
r= ()=+(2) +4) C8) +4 


where (2.12) has been used to evaluate (1/v)’. This can now be rearranged into 
the more convenient and memorisable form 


fi= (- 
v 
This can be expressed in words as the derivative of a quotient is equal to the bottom 


times the derivative of the top minus the top times the derivative of the bottom, all 
over the bottom squared. 


vu — ww’ 


—- (2.13) 


p2 


> Find the derivative with respect to x of f(x) = sinx/x. 


Using (2.13) with u(x) = sin x, v(x) = x and hence u'(x) = cos x, v'(x) = 1, we find 


xcosx—sinx cosx  sinx 


rs) = 


x x? * 


2.1.5 Implicit differentiation 


So far we have only differentiated functions written in the form y = f(x). 
However, we may not always be presented with a relationship in this simple 
form. As an example consider the relation x? — 3xy + y? = 2. In this case it is 
not possible to rearrange the equation to give y as a function of x. Nevertheless, 
by differentiating term by term with respect to x (implicit differentiation), we can 
find the derivative of y. 
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> Find dy/dx if x* —3xy+ y3 =2. 


Differentiating each term in the equation with respect to x we obtain 
d d 


; d , di3,_ a 
8) — Gx) + £0") = FQ), 


=> 3x7 (az + xy) +32 =0, 
where the derivative of 3xy has been found using the product rule. Hence, rearranging for 
dy/dx, 
dy y—x? 
dx yx 
Note that dy/dx is a function of both x and y and cannot be expressed as a function of x 
only. < 


2.1.6 Logarithmic differentiation 


In circumstances in which the variable with respect to which we are differentiating 
is an exponent, taking logarithms and then differentiating implicitly is the simplest 
way to find the derivative. 


> Find the derivative with respect to x of y =a*. 


To find the required derivative we first take logarithms and then differentiate implicitly: 


ld 
Iny =Ina* =xIna => oy elie, 
y dx 
Now, rearranging and substituting for y, we find 
dy 
= =ylna=a*Ina. <4 
dx 


2.1.7 Leibnitz’ theorem 


We have discussed already how to find the derivative of a product of two or more 
functions. We now consider Leibnitz’ theorem, which gives the corresponding 
results for the higher derivatives of products. 

Consider again the function f(x) = u(x)v(x). We know from the product rule 
that f’ = uv’ + u'v. Using the rule once more for each of the products, we obtain 


f" = (w" } u'v’) } (u'v’ } uv) 


= uv" + 2u'v' +u"v. 


Similarly, differentiating twice more gives 
f" = uv” + 3u'v" + 3u"'v' + uv, 
fF = w® + 4u'v!” + 6ulv" + 4ul"o! + uv. 
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The pattern emerging is clear and strongly suggests that the results generalise to 


n n 


f” = y n!} ype") = » "Cup, (2.14) 


“n—r)! 
= r\(n—r)! = 


where the fraction n!/[r!(n —r)!] is identified with the binomial coefficient "C, 
(see chapter 1). To prove that this is so, we use the method of induction as follows. 
Assume that (2.14) is valid for n equal to some integer N. Then 


fNTD = Sos (uy) 


= 3 NG. [uly NF) 4 ul Dy] 


N+1 
NG, yp! (N+1— i a CS pNtl- 8) 
s=1 


ll 
LM= 


ll 
oS 


where we have substituted summation index s for r in the first summation, and 
for r +1 in the second. Now, from our earlier discussion of binomial coefficients, 
equation (1.51), we have 


NG. SEN Guy = N+lC, 


and so, after separating out the first term of the first summation and the last 
term of the second, obtain 


N 
fry = NCgup At) +4 a NTC yy(Ntl-s) op NCyuNt Dy, 
s=1 


But NCy = 1 =Nt!Cp and NCy =1= N+1Cyi1, and so we may write 


N 
FOND = NAC yMyNHD 4 - NAIC ypNFI-9) 4 NH CY yD yO) 


s=1 


N+1 
= XS N+1 Coup Nts) 
s=0 


This is just (2.14) with n set equal to N + 1. Thus, assuming the validity of (2.14) 
for n = N implies its validity for n = N + 1. However, when n = 1 equation 
(2.14) is simply the product rule, and this we have already proved directly. These 
results taken together establish the validity of (2.14) for all n and prove Leibnitz’ 
theorem. 
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f(x) 


x 


Figure 2.2. A graph of a function, f(x), showing how differentiation corre- 
sponds to finding the gradient of the function at a particular point. Points B, 
Q and S are stationary points (see text). 


> Find the third derivative of the function f(x) = x* sinx. 


Using (2.14) we immediately find 


f'" (x) = 6sin x + 3(6x) cos x + 3(3x7)(— sin x) + x3(— cos x) 
= 3(2 — 3x) sinx + x(18 — x”) cosx. <4 


2.1.8 Special points of a function 


We have interpreted the derivative of a function as the gradient of the function at 
the relevant point (figure 2.1). If the gradient is zero for some particular value of 
x then the function is said to have a stationary point there. Clearly, in graphical 
terms, this corresponds to a horizontal tangent to the graph. 

Stationary points may be divided into three categories and an example of each 
is shown in figure 2.2. Point B is said to be a minimum since the function increases 
in value in both directions away from it. Point Q is said to be a maximum since 
the function decreases in both directions away from it. Note that B is not the 
overall minimum value of the function and Q is not the overall maximum; rather, 
they are a local minimum and a local maximum. Maxima and minima are known 
collectively as turning points. 

The third type of stationary point is the stationary point of inflection, S. In 
this case the function falls in the positive x-direction and rises in the negative 
x-direction so that S is neither a maximum nor a minimum. Nevertheless, the 
gradient of the function is zero at S, ie. the graph of the function is flat there, 
and this justifies our calling it a stationary point. Of course, a point at which the 
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gradient of the function is zero but the function rises in the positive x-direction 
and falls in the negative x-direction is also a stationary point of inflection. 

The above distinction between the three types of stationary point has been 
made rather descriptively. However, it is possible to define and distinguish sta- 
tionary points mathematically. From their definition as points of zero gradient, 
all stationary points must be characterised by df/dx = 0. In the case of the 
minimum, B, the slope, i.e. df/dx, changes from negative at A to positive at C 
through zero at B. Thus df /dx is increasing and so the second derivative d?f /dx* 
must be positive. Conversely, at the maximum, Q, we must have that d?f /dx? is 
negative. 

It is less obvious, but intuitively reasonable, that at S, d?f /dx? is zero. This may 
be inferred from the following observations. To the left of S the curve is concave 
upwards so that df /dx is increasing with x and hence df /dx? > 0. To the right 
of S, however, the curve is concave downwards so that df/dx is decreasing with 
x and hence d?f /dx? < 0. 

In summary, at a stationary point df/dx =0 and 


(i) for a minimum, d?f /dx* > 0, 
(ii) for a maximum, d?f /dx? < 0, 


(iii) for a stationary point of inflection, d*f /dx? = 0 and d’f /dx? changes sign 
through the point. 


In case (iii), a stationary point of inflection, in order that d*f /dx* changes sign 
through the point we normally require d*f/dx? # 0 at that point. This simple 
rule can fail for some functions, however, and in general if the first non-vanishing 
derivative of f(x) at the stationary point is f then if n is even the point is a 
maximum or minimum and if n is odd the point is a stationary point of inflection. 
This may be seen from the Taylor expansion (see equation (4.17)) of the function 
about the stationary point, but it is not proved here. 


> Find the positions and natures of the stationary points of the function 
f(x) = 2x? — 3x? — 36x 4 2. 


The first criterion for a stationary point is that df /dx = 0, and hence we set 


d 
af = 6x" — 6x — 36 = 0, 
dx 
from which we obtain 
(x — 3)(x +2) =0. 
Hence the stationary points are at x = 3 and x = —2. To determine the nature of the 
stationary point we must evaluate df /dx*: 


af 
aa 12x — 6. 
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F(x) 


x 


Figure 2.3. The graph of a function f(x) that has a general point of inflection 
at the point G. 


Now, we examine each stationary point in turn. For x = 3, d?f /dx? = 30. Since this is 
positive, we conclude that x = 3 is a minimum. Similarly, for x = —2, d°f /dx* = —30 and 
so x = —2 is a maximum. < 


So far we have concentrated on stationary points, which are defined to have 
df /dx = 0. We have found that at a stationary point of inflection d?f /dx? is 
also zero and changes sign. This naturally leads us to consider points at which 
d?f /dx? is zero and changes sign but at which df /dx is not, in general, zero. Such 
points are called general points of inflection or simply points of inflection. Clearly, 
a stationary point of inflection is a special case for which df/dx is also zero. 
At a general point of inflection the graph of the function changes from being 
concave upwards to concave downwards (or vice versa), but the tangent to the 
curve at this point need not be horizontal. A typical example of a general point 
of inflection is shown in figure 2.3. 

The determination of the stationary points of a function, together with the 
identification of its zeros, infinities and possible asymptotes, is usually sufficient 
to enable a graph of the function showing most of its significant features to be 
sketched. Some examples for the reader to try are included in the exercises at the 
end of this chapter. 


2.1.9 Curvature of a function 


In the previous section we saw that at a point of inflection of the function 
f(x), the second derivative d?f/dx? changes sign and passes through zero. The 
corresponding graph of f shows an inversion of its curvature at the point of 
inflection. We now develop a more quantitative measure of the curvature of a 
function (or its graph), which is applicable at general points and not just in the 
neighbourhood of a point of inflection. 

As in figure 2.1, let 0 be the angle made with the x-axis by the tangent at a 
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F(x) 


“<9 oS O+A0 


x 


Figure 2.4 Two neighbouring tangents to the curve f(x) whose slopes differ 
by AO. The angular separation of the corresponding radii of the circle of 
curvature is also A@. 


point P on the curve f = f(x), with tan 0 = df/dx evaluated at P. Now consider 
also the tangent at a neighbouring point Q on the curve, and suppose that it 
makes an angle 0 + A@ with the x-axis, as illustrated in figure 2.4. 

It follows that the corresponding normals at P and Q, which are perpendicular 
to the respective tangents, also intersect at an angle AQ. Furthermore, their point 
of intersection, C in the figure, will be the position of the centre of a circle that 
approximates the arc PQ, at least to the extent of having the same tangents at 
the extremities of the arc. This circle is called the circle of curvature. 

For a finite arc PQ, the lengths of CP and CQ will not, in general, be equal, 
as they would be if f = f(x) were in fact the equation of a circle. But, as Q 
is allowed to tend to P, ie. as AO — 0, they do become equal, their common 
value being p, the radius of the circle, known as the radius of curvature. It follows 
immediately that the curve and the circle of curvature have a common tangent 
at P and lie on the same side of it. The reciprocal of the radius of curvature, p~', 
defines the curvature of the function f(x) at the point P. 

The radius of curvature can be defined more mathematically as follows. The 
length As of arc PQ is approximately equal to pA@ and, in the limit A@ — 0, this 
relationship defines p as 


As ds 

= lim — = —. 2.15 
P= 0000 dO (2) 
It should be noted that, as s increases, 0 may increase or decrease according to 
whether the curve is locally concave upwards (i.e. shaped as if it were near a 
minimum in f(x)) or concave downwards. This is reflected in the sign of p, which 
therefore also indicates the position of the curve (and of the circle of curvature) 
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relative to the common tangent, above or below. Thus a negative value of p 
indicates that the curve is locally concave downwards and that the tangent lies 
above the curve. 

We next obtain an expression for p, not in terms of s and @ but in terms 
of x and f(x). The expression, though somewhat cumbersome, follows from the 
defining equation (2.15), the defining property of 0 that tan@ = df/dx = f’ and 
the fact that the rate of change of arc length with x is given by 


~ 2711/2 
1+ (4) | F (2.16) 
dx 


This last result, simply quoted here, is proved more formally in subsection 2.2.13. 
From the chain rule (2.11) it follows that 


ds ds dx 


Differentiating both sides of tan0 = df /dx with respect to x gives 


from which, using sec? 0 = 1 + tan? @ = 1+4(f’)?, we can obtain dx/d0 as 


dx 1+tan?@  14(f') 
dé id f" 


(2.18) 


Substituting (2.16) and (2.18) into (2.17) then yields the final expression for p, 


1513/2 
p= ao (2.19) 


It should be noted that the quantity in brackets is always positive and that its 
three-halves root is also taken as positive. The sign of p is thus solely determined 
by that of d?f/dx?, in line with our previous discussion relating the sign to 
whether the curve is concave or convex upwards. If, as happens at a point of 
inflection, d?f /dx* is zero then p is formally infinite and the curvature of f(x) is 
zero. As d’f /dx? changes sign on passing through zero, both the local tangent 
and the circle of curvature change from their initial positions to the opposite side 
of the curve. 
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> Show that the radius of curvature at the point (x,y) on the ellipse 


=1 


Ge” Le 


has magnitude (a*y* + b*x?)?/?/(a*b*) and the opposite sign to y. Check the special case 


b =a, for which the ellipse becomes a circle. 


and so 


A second differentiation, using (2.13), then yields 


@y BR fy—x'\_ bt /y? x) bt 
ae Oe y? ys rar ys’ 
where we have used the fact that (x, y) lies on the ellipse. We note that d*y/dx*, and hence 


p, has the opposite sign to y? and hence to y. Substituting in (2.19) gives for the magnitude 
of the radius of curvature 


[1 + b4x?/(a*y?)| 372 
—b4/(ay3) 


For the special case b = a, |p| reduces to a~*(y? + x?)*/? and, since x? + y? = a’, this in 
turn gives |p| =a, as expected. < 


(aty? + b4x2)3/2 


lpl= re 


The discussion in this section has been confined to the behaviour of curves 
that lie in one plane; examples of the application of curvature to the bending of 
loaded beams and to particle orbits under the influence of a central forces can be 
found in the exercises at the ends of later chapters. A more general treatment of 
curvature in three dimensions is given in section 10.3, where a vector approach is 
adopted. 


2.1.10 Theorems of differentiation 


Rolle’s theorem 


Rolle’s theorem (figure 2.5) states that if a function f(x) is continuous in the 
range a< x <c, is differentiable in the range a < x <c and satisfies f(a) = f(c) 
then for at least one point x = b, where a < b < c, f'(b) = 0. Thus Rolle’s 
theorem states that for a well-behaved (continuous and differentiable) function 
that has the same value at two points either there is at least one stationary point 
between those points or the function is a constant between them. The validity of 
the theorem is immediately apparent from figure 2.5 and a full analytic proof will 
not be given. The theorem is used in deriving the mean value theorem, which we 
now discuss. 
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Figure 2.5 The graph of a function f(x), showing that if f(a) = f(c) then at 
one point at least between x = a and x = c the graph has zero gradient. 


Figure 2.6 The graph of a function f(x); at some point x = b it has the same 
gradient as the line AC. 


Mean value theorem 


The mean value theorem (figure 2.6) states that if a function f(x) is continuous 
in the range a < x <c and differentiable in the range a < x <c then 


7 LO -f@ 
ro=- (2.20) 
for at least one value b where a < b < c. Thus the mean value theorem states 
that for a well-behaved function the gradient of the line joining two points on the 
curve is equal to the slope of the tangent to the curve for at least one intervening 
point. 
The proof of the mean value theorem is found by examination of figure 2.6, as 
follows. The equation of the line AC is 


g(x) = f(a) + (x— yO, 
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and hence the difference between the curve and the line is 


fle) fa) 


c—a 


h(x) = f(x) — g(x) = f(x) — f(a) — (x — a) 


Since the curve and the line intersect at A and C, h(x) = 0 at both of these points. 
Hence, by an application of Rolle’s theorem, h'(x) = 0 for at least one point b 
between A and C. Differentiating our expression for h(x), we find 


W(x) = x) - LOE), 
and hence at b, where h'(x) = 0, 
y= 10-10, 


Applications of Rolle’s theorem and the mean value theorem 


Since the validity of Rolle’s theorem is intuitively obvious, given the conditions 
imposed on f(x), it will not be surprising that the problems that can be solved 
by applications of the theorem alone are relatively simple ones. Nevertheless we 
will illustrate it with the following example. 


> What semi-quantitative results can be deduced by applying Rolle’s theorem to the fol- 
lowing functions f(x), with a and c chosen so that f(a) = f(c) = 0? (i) sinx, (ii) cos x, 
(ili)x? — 3x + 2, (iv) x7 + 7x + 3, (v) 2x3 — 9x? — 24x + k. 


(i) If the consecutive values of x that make sinx = 0 are 0, %2,... (actually x = nz, for 
any integer n) then Rolle’s theorem implies that the derivative of sinx, namely cos x, has 
at least one zero lying between each pair of values «; and a+. 

(ii) In an exactly similar way, we conclude that the derivative of cos x, namely — sin x, 
has at least one zero lying between consecutive pairs of zeros of cosx. These two re- 
sults taken together (but neither separately) imply that sinx and cosx have interleaving 
Zeros. 

(iii) For f(x) = x? —3x +2, f(a) = f(c) = 0 if a and c are taken as 1 and 2 respectively. 
Rolle’s theorem then implies that f’(x) = 2x — 3 = 0 has a solution x = b with b in the 
range 1 < b < 2. This is obviously so, since b = 3/2. 

(iv) With f(x) = x? + 7x +3, the theorem tells us that if there are two roots of 
x? +7x +3 = 0 then they have the root of f’(x) = 2x +7 = 0 lying between them. Thus if 
there are any (real) roots of x? + 7x + 3 = 0 then they lie one on either side of x = —7/2. 
The actual roots are (—7 + J/37)/2. 

(v) If f(x) = 2x3 — 9x? — 24x +k then f(x) = 0 is the equation 6x? — 18x — 24 = 0, 
which has solutions x = —1 and x = 4. Consequently, if « and « are two different roots 
of f(x) = 0 then at least one of —1 and 4 must lie in the open interval a to a. If, as is 
the case for a certain range of values of k, f(x) = 0 has three roots, «1,%. and «3, then 
a<—-l<m<4<0. 
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In each case, as might be expected, the application of Rolle’s theorem does no more than 
focus attention on particular ranges of values; it does not yield precise answers. < 


Direct verification of the mean value theorem is straightforward when it is 
applied to simple functions. For example, if f(x) = x’, it states that there is a 
value b in the interval a < b <c such that 


© —a@ = f(c)— f(a) = (c —a)f'(b) = (c — 2b. 


This is clearly so, since b = (a+ c)/2 satisfies the relevant criteria. 

As a slightly more complicated example we may consider a cubic equation, say 
f(x) = x3 + 2x? + 4x — 6 = 0, between two specified values of x, say 1 and 2. In 
this case we need to verify that there is a value of x lying in the range 1 << x < 2 
that satisfies 


18—1 = f(2)—f(1) = (2— f(x) = 13x? +. 4x +4). 


This is easily done, either by evaluating 3x*+4x-+4—17 at x = 1 and at x = 2 and 
checking that the values have opposite signs or by solving 3x7 + 4x +4—17=0 
and showing that one of the roots lies in the stated interval. 

The following applications of the mean value theorem establish some general 
inequalities for two common functions. 


> Determine inequalities satisfied by Inx and sin x for suitable ranges of the real variable x. 


1 


Since for positive values of its argument the derivative of Inx is x~’, the mean value 


theorem gives us 
Inc—Ina_ 1 


c-a b 


for some b in 0 < a < b <c. Further, since a < b <c implies that c~! < b7' < a“', we 


have 
1 Inc—Ina 1 
= < ——— <., 
c c-a a 


or, multiplying through by c—a and writing c/a = x where x > 1, 
1 
1—-<Inx<x-l. 
x 


Applying the mean value theorem to sin x shows that 


sinc — sina 
—— =cosb 
ca 
for some b lying between a and c. If a and c are restricted to lie in the rangeO <a<c<q, 
in which the cosine function is monotonically decreasing (i.e. there are no turning points), 
we can deduce that 
sinc — sina 
cosc < ———— < cosa. 4 
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f(x) 


Figure 2.7 An integral as the area under a curve. 


2.2 Integration 


The notion of an integral as the area under a curve will be familiar to the reader. 
In figure 2.7, in which the solid line is a plot of a function f(x), the shaded area 
represents the quantity denoted by 


b 
I -| f (x) dx. (2.21) 


This expression is known as the definite integral of f(x) between the lower limit 
x =a and the upper limit x = b, and f(x) is called the integrand. 


2.2.1 Integration from first principles 


The definition of an integral as the area under a curve is not a formal definition, 
but one that can be readily visualised. The formal definition of I involves 
subdividing the finite interval a < x < b into a large number of subintervals, by 
defining intermediate points €; such that a = €&) < €) <&; <---< &, =b, and 
then forming the sum 


S= So f(iG— G4), (2.22) 
i=1 


where x; is an arbitrary point that lies in the range €;_; < x; < &; (see figure 2.8). 
If now a is allowed to tend to infinity in any way whatsoever, subject only to the 
restriction that the length of every subinterval ¢;; to €; tends to zero, then S 
might, or might not, tend to a unique limit, J. If it does then the definite integral 
of f(x) between a and b is defined as having the value I. If no unique limit exists 
the integral is undefined. For continuous functions and a finite interval a< x <b 


the existence of a unique limit is assured and the integral is guaranteed to exist. 
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F(x) 


; 
a Xig; X2 & X3&3 X4 fg Xs Ob x 


Figure 2.8 The evaluation of a definite integral by subdividing the interval 
a <x <b into subintervals. 


> Evaluate from first principles the integral I = ie x? dx. 


We first approximate the area under the curve y = x” between 0 and b by n rectangles of 
equal width h. If we take the value at the lower end of each subinterval (in the limit of an 
infinite number of subintervals we could equally well have chosen the value at the upper 
end) to give the height of the corresponding rectangle, then the area of the kth rectangle 
will be (kh)?h = k?h?. The total area is thus 


n—1 


A=So Ph = (i )gn(n— 1)(2n— 1), 
k=0 


where we have used the expression for the sum of the squares of the natural numbers 
derived in subsection 1.7.1. Now h = b/n and so 


b\ n b3 1 1 
r (=) a(n 1)2n—=1) z (1 -) (2 *): 


As n— «, A > b3/3, which is thus the value I of the integral. < 


Some straightforward properties of definite integrals that are almost self-evident 
are as follows: 


b a 
| Odx =0, | f(x)dx =0, (2.23) 
c b c 
(x) dx = d d 2.24 
[ serax= [ tooas+ [ foods, (2.24) 
b b b 
if [f(x) + g(x)] ax = [ foodx+ [ g(x) dx. (2.25) 
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Combining (2.23) and (2.24) with c set equal to a shows that 


b a 
ip f(x) dx = -{ f(x) dx. (2.26) 
a b 


2.2.2 Integration as the inverse of differentiation 


The definite integral has been defined as the area under a curve between two 
fixed limits. Let us now consider the integral 


F(x) = i sau (2.27) 


in which the lower limit a remains fixed but the upper limit x is now variable. It 
will be noticed that this is essentially a restatement of (2.21), but that the variable 
x in the integrand has been replaced by a new variable u. It is conventional to 
rename the dummy variable in the integrand in this way in order that the same 
variable does not appear in both the integrand and the integration limits. 

It is apparent from (2.27) that F(x) is a continuous function of x, but at first 
glance the definition of an integral as the area under a curve does not connect with 
our assertion that integration is the inverse process to differentiation. However, 
by considering the integral (2.27) and using the elementary property (2.24), we 
obtain 


x+Ax 
Fix-+Ax) = f i f(u)du 
x x+Ax 
=f soydus [fod 


x+Ax 
= F(x) + : f(u) du. 


Rearranging and dividing through by Ax yields 


F(x +Ax)—F(x) 1 pt 
ro Ke | f(u) du. 


Letting Ax — 0 and using (2.1) we find that the LHS becomes dF /dx, whereas 
the RHS becomes f(x). The latter conclusion follows because when Ax is small 
the value of the integral on the RHS is approximately f(x)Ax, and in the limit 
Ax — 0 no approximation is involved. Thus 
dF (x) 
dx 
or, substituting for F(x) from (2.27), 


7 | | “fo au = flo) 
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From the last two equations it is clear that integration can be considered as 
the inverse of differentiation. However, we see from the above analysis that the 
lower limit a is arbitrary and so differentiation does not have a unique inverse. 
Any function F(x) obeying (2.28) is called an indefinite integral of f(x), though 
any two such functions can differ by at most an arbitrary additive constant. Since 
the lower limit is arbitrary, it is usual to write 


F(x) = ii ; f(u) du (2.29) 


and explicitly include the arbitrary constant only when evaluating F(x). The 
evaluation is conventionally written in the form 


/ f(x) dx = F(x) + (2.30) 


where c is called the constant of integration. It will be noticed that, in the absence 
of any integration limits, we use the same symbol for the arguments of both f 
and F. This can be confusing, but is sufficiently common practice that the reader 
needs to become familiar with it. 

We also note that the definite integral of f(x) between the fixed limits x = a 
and x = b can be written in terms of F(x). From (2.27) we have 


b b a 
i f(x)dx = f(x) ax— | f (x) dx 
= F(b)— F(a), (2.31) 


where Xo is any third fixed point. Using the notation F’(x) = dF/dx, we may 
rewrite (2.28) as F’(x) = f(x), and so express (2.31) as 


b 
/ F'(x) dx = F(b) — F(a) = [F/’. 


In contrast to differentiation, where repeated applications of the product rule 
and/or the chain rule will always give the required derivative, it is not always 
possible to find the integral of an arbitrary function. Indeed, in most real phys- 
ical problems exact integration cannot be performed and we have to revert to 
numerical approximations. Despite this cautionary note, it is in fact possible to 
integrate many simple functions and the following subsections introduce the most 
common types. Many of the techniques will be familiar to the reader and so are 
summarised by example. 


2.2.3 Integration by inspection 


The simplest method of integrating a function is by inspection. Some of the more 
elementary functions have well-known integrals that should be remembered. The 
reader will notice that these integrals are precisely the inverses of the derivatives 
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found near the end of subsection 2.1.1. A few are presented below, using the form 
given in (2.30): 


ax't! 
[ad =axte, [otax= +¢, 
n 


Ax 
a x 


[ acosbx dx = asm +c, [asinbdx = —acoshx +6, 


_ antl 
[ atan beds = ERY 4, J eosin" bx dx = 7 +o 
b b(n + 1) 


=) n+1 bx 
/ 7 dx = tan7! (=) LC, J asinbxeos bx dx = OOS a ep CG 
a? + x? a b(n +1) 


1 


= - 1 {XX 
x = COs dx = sin —)t+6 
a 


—1 d (*) ig / 1 

[a2 — x2 a : [a2 — x2 
where the integrals that depend on n are valid for all n 4 —1 and where a and b 
are constants. In the two final results |x| < a. 


2.2.4 Integration of sinusoidal functions 


Integrals of the type f sin’xdx and f cos’xdx may be found by using trigono- 
metric expansions. Two methods are applicable, one for odd n and the other for 
even n. They are best illustrated by example. 


> Evaluate the integral I = sin? x dx. 


Rewriting the integral as a product of sinx and an even power of sinx, and then using 
the relation sin? x = 1 — cos? x yields 


1 = f sint xsinxdx 
= (1 cos? x) sin xdx 
= [ (12008 x + cos" x)sin xd 


= Joins —2sin x cos’ x + sin x cos* x) dx 


=—cosx + 3cos’x— tos’ x +.¢, 


where the integration has been carried out using the results of subsection 2.2.3. < 
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> Evaluate the integral I = f cos* x dx. 


Rewriting the integral as a power of cos*x and then using the double-angle formula 
cos? x = $(1 + cos 2x) yields 


2 
r= f(cos’xtax= { (+5°*) dx 


= i z(1 + 2.cos 2x + cos 2x) dx. 


Using the double-angle formula again we may write cos? 2x = (1 + cos 4x), and hence 


I = [7 + $.cos2x + 3(1 + cos4x)] dx 
= gx t 4 sin 2x + axt % sin4x +¢ 
= ext fsin2x+ $sin4x +c. < 


2.2.5 Logarithmic integration 


Integrals for which the integrand may be written as a fraction in which the 
numerator is the derivative of the denominator may be evaluated using 


Fe) dx = Inf(x) +c. (2.32) 
f(x) 
This follows directly from the differentiation of a logarithm as a function of a 
function (see subsection 2.1.3). 


> Evaluate the integral 


2» 
{= + 2cosx 5 


x3 + sinx 


We note first that the numerator can be factorised to give 2(3x? + cos x), and then that 
the quantity in brackets is the derivative of the denominator. Hence 


2 
I 2/f > ERE 2In(x? + sinx) +c. < 
x? + sin x 


2.2.6 Integration using partial fractions 


The method of partial fractions was discussed at some length in section 1.4, but 
in essence consists of the manipulation of a fraction (here the integrand) in such 
a way that it can be written as the sum of two or more simpler fractions. Again 
we illustrate the method by an example. 
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> Evaluate the integral 


We note that the denominator factorises to give x(x + 1). Hence 


1 
=f—* 


We now separate the fraction into two partial fractions and integrate directly: 


1 1 x 
J —_— | 
I IG 7 :) dx =Inx —In(x+1)+c in(- :) +c <4 


2.2.7 Integration by substitution 


Sometimes it is possible to make a substitution of variables that turns a com- 
plicated integral into a simpler one, which can then be integrated by a standard 
method. There are many useful substitutions and knowing which to use is a matter 
of experience. We now present a few examples of particularly useful substitutions. 


> Evaluate the integral 


1 
f= dx. 
= . 


Making the substitution x = sinu, we note that dx = cosudu, and hence 


du=u-+c. 


I -| mews -| Seg cosudu= | 
Now substituting back for u, 
I=sin'x+e. 
This corresponds to one of the results given in subsection 2.2.3. < 


Another particular example of integration by substitution is afforded by inte- 
grals of the form 


1 1 


In these cases, making the substitution t = tan(x/2) yields integrals that can 
be solved more easily than the originals. Formulae expressing sinx and cosx in 
terms of t were derived in equations (1.32) and (1.33) (see p. 14), but before we 
can use them we must relate dx to dt as follows. 


65 


PRELIMINARY CALCULUS 


Since 
dt 1: pe 9 2x\_ 140 
Fe 77 FH 5 (1+ tan’ 5) = 
the required relationship is 
dx = dt (2.34) 
t= 7 pt : 


> Evaluate the integral 


t= f ix 
1+3cosx 


Rewriting cos x in terms of t and using (2.34) yields 


2 2 

} (Feces esos (3) Ht 
2(1 + 1?) 2 F 
-[z Wares : 
2 

ee = (a (arn 

1 
= a(a5 tam) . 


: In(./2 —t)4 qin? +t) +e 
/2 + tan (x/2) 
./2 — tan (x/2) 


+c <4 


an 


Integrals of a similar form to (2.33), but involving sin 2x, cos 2x, tan 2x, sin? x, 
cos? x or tan?x instead of cosx and sinx, should be evaluated by using the 
substitution t = tan x. In this case 


sin x = : > Cosx= : and dx= nh (2.35) 


J1 +P 14+? 


A final example of the evaluation of integrals using substitution is the method 
of completing the square (cf. subsection 1.7.3). 
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> Evaluate the integral 


1 
cy ne 
oer ‘i 


We can write the integral in the form 


1 
I= | ——— x. 
/ +243. 
Substituting y = x + 2, we find dy = dx and hence 


1 
1=|—<d 
lw y, 


Hence, by comparison with the table of standard integrals (see subsection 2.2.3) 


rs B tan (=) be B tan () +e 


2.2.8 Integration by parts 


Integration by parts is the integration analogy of product differentiation. The 
principle is to break down a complicated function into two functions, at least one 
of which can be integrated by inspection. The method in fact relies on the result 
for the differentiation of a product. Recalling from (2.6) that 


d ive dv . du 
dx > “ox dx” 


where u and v are functions of x, we now integrate to find 


dv du 
w= fu ax + | v dx. 
dx dx 


Rearranging into the standard form for integration by parts gives 


dv du 
pes dx =w— | Mode. (2.36) 


Integration by parts is often remembered for practical purposes in the form 
the integral of a product of two functions is equal to {the first times the integral of 
the second} minus the integral of {the derivative of the first times the integral of 
the second}. Here, u is ‘the first’ and dv/dx is ‘the second’; clearly the integral v 
of ‘the second’ must be determinable by inspection. 


> Evaluate the integral I = [ x sinx dx. 


In the notation given above, we identify x with u and sin x with dv/dx. Hence v = —cosx 
and du/dx = 1 and so using (2.36) 


I = x(—cos x) fon cosx)dx =—xcosx+sinx+c. < 
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The separation of the functions is not always so apparent, as is illustrated by 
the following example. 


> Evaluate the integral I = eer dx. 


Firstly we rewrite the integral as 
I -f# (xe) dx. 


Now, using the notation given above, we identify x? with u and xe with dv /dx. Hence 
v= —te and du/dx = 2x, so that 


—x2 
e +c <4 


Nis 


Tie 2e x —x? 1\.2,,—x2 
I xe (—x)e™ dx 5xe 


A trick that is sometimes useful is to take ‘1’ as one factor of the product, as 
is illustrated by the following example. 


> Evaluate the integral I = f Inx dx. 


Firstly we rewrite the integral as 


I = fons tax 


Now, using the notation above, we identify Inx with u and 1 with dv/dx. Hence we have 
v =x and du/dx = 1/x, and so 


vi = (in [ (=) sax =sinx— xe < 


It is sometimes necessary to integrate by parts more than once. In doing so, 
we may occasionally re-encounter the original integral J. In such cases we can 
obtain a linear algebraic equation for J that can be solved to obtain its value. 


> Evaluate the integral I = { e* cos bx dx. 


Integrating by parts, taking e* as the first function, we find 


fae (=) [oe (=) be 


where, for convenience, we have omitted the constant of integration. Integrating by parts 
a second time, 


ax [ sin bx ax [ — C08 bx 2 ax [ — C08 bx 
IT=e D — ae pS + ae pr dx. 


Notice that the integral on the RHS is just —a*/b? times the original integral J. Thus 


a halle a e 
IT=e (jsinbx-+ fcosbx) — Fr, 
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Rearranging this expression to obtain J explicitly and including the constant of integration 


we find 


mea sin bx + acos bx) +c. (2.37) 
a 


Another method of evaluating this integral, using the exponential of a complex number, 
is given in section 3.6. < 


2.2.9 Reduction formulae 


Integration using reduction formulae is a process that involves first evaluating a 
simple integral and then, in stages, using it to find a more complicated integral. 


> Using integration by parts, find a relationship between I, and I,-1 where 


1 
I, = i (l=x?)"dx 
0 


: ce 1 
and n is any positive integer. Hence evaluate I, = f, (1 — x*)* dx. 


Writing the integrand as a product and separating the integral into two we find 


1 
I, = | (1—38)(1 — x3)" dx 
0 


1 1 
u (1—x3)""! dx | (1 — x3)""! dx. 
0 0 


The first term on the RHS is clearly I,,_; and so, writing the integrand in the second term 
on the RHS as a product, 


1 
I, = Th-1 = i (x)x?(1 = xeyrt dx. 
0 


Integrating by parts we find 


1 1 
h=hat[za-xy],- [ =(1— x3)" dx 
0 


3n 


=I,1+ 0- awl 
n 


which on rearranging gives 
3n 
aoe: 
3n+1 
We now have a relation connecting successive integrals. Hence, if we can evaluate Ip, we 
can find I,, I, etc. Evaluating Jo is trivial: 


1 1 
Io fa x3)? dx fa [x]} = 1. 
0 0 


_ Gxt) it _ @x2) Coe 2 
Bxh+i> # 7" Bx2V+1° 4° 4 
Although the first few I, could be evaluated by direct multiplication, this becomes tedious 
for integrals containing higher values of n; these are therefore best evaluated using the 
reduction formula. < 


I, = 


Hence 
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2.2.10 Infinite and improper integrals 


The definition of an integral given previously does not allow for cases in which 
either of the limits of integration is infinite (an infinite integral) or for cases 
in which f(x) is infinite in some part of the range (an improper integral), e.g. 
f(x) = (2— x)" near the point x = 2. Nevertheless, modification of the 
definition of an integral gives infinite and improper integrals each a meaning. 

In the case of an integral I = [ i f(x) dx, the infinite integral, in which b tends 
to 0, is defined by 


* b 


As previously, F(x) is the indefinite integral of f(x) and limp_,.. F(b) means the 
limit (or value) that F(b) approaches as b > oo; it is evaluated after calculating 
the integral. The formal concept of a limit will be introduced in chapter 4. 


> Evaluate the integral 


22 x 
r= f eae 


Integrating, we find F(x) = —}(x? +.a’)"! +c and so 


T=li -1 —1 1 4 
fae 2(b2 + a?) 2a? 2a" 
For the case of improper integrals, we adopt the approach of excluding the 


unbounded range from the integral. For example, if the integrand f(x) is infinite 
at x =c (say),a<c<b then 


b c—4 b 
| f(x) dx = tim, | f(x) dx + lim ie f(x) dx. 


> Evaluate the integral I = fe2 — x)"/4 dx. 


Integrating directly, 


I = lim [—$(2— x)*/"] ° = lim [$e] + $2°* = (4) 24. 


e0 


2.2.11 Integration in plane polar coordinates 


In plane polar coordinates p,@, a curve is defined by its distance p from the 
origin as a function of the angle @ between the line joining a point on the curve 
to the origin and the x-axis, ic. p = p(@). The area of an element is given by 
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O > 


Figure 2.9 Finding the area of a sector OBC defined by the curve p(#) and 
the radii OB, OC, at angles to the x-axis $1, ¢2 respectively. 


dA = 5p? d@, as illustrated in figure 2.9, and hence the total area between two 
angles d; and ¢ is given by 


go 
A= / 5p° dd. (2.38) 
g 


1 


An immediate observation is that the area of a circle of radius a is given by 


2n 
A= | sa dp = [Lao]. = na’. 
0 


> The equation in polar coordinates of an ellipse with semi-axes a and b is 


43 
1 cos*@ sin 
a pe” 


Find the area A of the ellipse. 


Using (2.38) and symmetry, we have 


1p? arb? ale 1 
a=>/ a ap= 20? [ —________ dg. 
2Jo bcos?h +a? sin’ ob 0 bcos? b+ a? sin’ 


To evaluate this integral we write t = tan@ and use (2.35): 


iy 84 oo 4 
= 2972 2 
A= 2a’b [ Frge aw i: Barret 


Finally, from the list of standard integrals (see subsection 2.2.3), 


ee endl coer 2 qo) — 
A=2b wo aa|, 728° (5 0) = nab. « 


71 


PRELIMINARY CALCULUS 


2.2.12 Integral inequalities 


Consider the functions f(x), ¢1(x) and 2(x) such that ¢1(x) < f(x) < ¢2(x) for 
all x in the range a < x < b. It immediately follows that 


b b b 
J dwoars [ pordx< [ dxsas, (2.39) 
which gives us a way of estimating an integral that is difficult to evaluate explicitly. 


> Show that the value of the integral 


1 
1 
r= (14202 +x3)'2 dx 


lies between 0.810 and 0.882. 


We note that for x in the range 0 < x < 1,0 <x? < x. Hence 


(1 4+x2)"2 <1 42 4 8)¥2 < (1 4 2x2), 
and so 
1 ey 1 es 1 
(+22 ~ 1432 48)? ~ T4222 
Consequently, 


E J d = d d th : d. 
» +x” =f, +e Feye =f. C+ 22 
from which we obtain 
1 er 
[nx + VI+>)] 27> lam («+ /4+)| 


0.8814 > I > 0.8105 
0.882 > I > 0.810. 


1 


0 


In the last line the calculated values have been rounded to three significant figures, 
one rounded up and the other rounded down so that the proved inequality cannot be 
unknowingly made invalid. < 


2.2.13 Applications of integration 


Mean value of a function 
The mean value m of a function between two limits a and b is defined by 


b 
m= — : f (x) dx. (2.40) 
The mean value may be thought of as the height of the rectangle that has the 
same area (over the same interval) as the area under the curve f(x). This is 
illustrated in figure 2.10. 
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Figure 2.10 The mean value m of a function. 


> Find the mean value m of the function f(x) = x? between the limits x = 2 and x = 4. 


Using (2.40), 


Finding the length of a curve 


Finding the area between a curve and certain straight lines provides one example 
of the use of integration. Another is in finding the length of a curve. If a curve 
is defined by y = f(x) then the distance along the curve, As, that corresponds to 
small changes Ax and Ay in x and y is given by 


As = \/ (Ax)? + (Ay)?; (2.41) 


this follows directly from Pythagoras’ theorem (see figure 2.11). Dividing (2.41) 
through by Ax and letting Ax > 0 we obtain’ 


ds dy 3 
—=,/1 —). 
dx r (2) 


Clearly the total length s of the curve between the points x = a and x = b is then 
given by integrating both sides of the equation: 


b | 2 
s= / 1+ () dx. (2.42) 


S Instead of considering small changes Ax and Ay and letting these tend to zero, we could have 
derived (2.41) by considering infinitesimal changes dx and dy from the start. After writing (ds) 
(dx)? + (dy), (2.41) may be deduced by using the formal device of dividing through by dx. Although 
not mathematically rigorous, this method is often used and generally leads to the correct result. 
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F(x) 


As A y 
Ax 


x 


Figure 2.11 The distance moved along a curve, As, corresponding to the 
small changes Ax and Ay. 


In plane polar coordinates, 


r 2 
ds=V/(dr? +(rdd? > s= iS 1+r (4) dr. 


ry 


> Find the length of the curve y = x*/? from x = 0 to x = 2. | 


Using (2.42) and noting that dy/dx = 3x, the length s of the curve is given by 


Surfaces of revolution 


Consider the surface S formed by rotating the curve y = f(x) about the x-axis 
(see figure 2.12). The surface area of the ‘collar’ formed by rotating an element 
of the curve, ds, about the x-axis is 2zy ds, and hence the total surface area is 


b 
s- | 2ry ds. 


Since (ds)? = (dx)? + (dy) from (2.41), the total surface area between the planes 


x =aand x =b is 
b dy 2 
s= | any + (2) dx. (2.44) 
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Figure 2.12 The surface and volume of revolution for the curve y = f(x). 


> Find the surface area of a cone formed by rotating about the x-axis the line y = 2x 
between x = 0 and x =h. 


Using (2.44), the surface area is given by 


2 


h 
s= | (27)2x4/1+ 2%) dx 
h 
= f anx(1+2)'? de = [ 4V5nxas 
= [25m] = 2,/5n(h? — 0) = 2\/5ah?. « 


We note that a surface of revolution may also be formed by rotating a line 
about the y-axis. In this case the surface area between y = a and y = b is 


(*) 
oy +( oO) ay. (2.45) 


Volumes of revolution 


The volume V enclosed by rotating the curve y = f(x) about the x-axis can also 
be found (see figure 2.12). The volume of the disc between x and x + dx is given 
by dV = my? dx. Hence the total volume between x = a and x = b is 


b 
V= / my? dx. (2.46) 
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> Find the volume of a cone enclosed by the surface formed by rotating about the x-axis 
the line y = 2x between x = 0 and x =h. 


Using (2.46), the volume is given by 


h h 
V= | n(2x) dx = i: Anx? dx 
0 0 


[4nx*]} = 4n(h — 0) = $a’. « 


As before, it is also possible to form a volume of revolution by rotating a curve 
about the y-axis. In this case the volume enclosed between y = a and y = b is 


b 
v= f nxtdy, (2.47) 


2.3 Exercises 
2.1 Obtain the following derivatives from first principles: 


(a) the first derivative of 3x + 4; 
(b) the first, second and third derivatives of x? + x; 
(c) the first derivative of sin x. 


22 Find from first principles the first derivative of (x +3)? and compare your answer 
with that obtained using the chain rule. 
233) Find the first derivatives of 


(a) x? exp x, (b) 2sinxcos x, (c) sin 2x, (d) x sin ax, 
(e) (expax)(sin ax) tan~! ax, (f) In(x* + x~*), 
(g) In(a* + a™), (hh) x*. 
2.4 Find the first derivatives of 
(a) x/(a+x), (b) x/(1 —x)!/?, (c) tan x, as sin x/ cos x, 
(d) (3x? + 2x + 1)/(8x? — 4x + 2). 


2.5 Use result (2.12) to find the first derivatives of 
(a) (2x +3)-, (b) sec? x, (c) cosech?3x, (d) 1/Inx, (e) 1/[sin7!(x/a)]. 
2.6 Show that the function y(x) = exp(—|x|) defined by 
exp x for x < 0, 
y(x) = 41 for x = 0, 


exp(—x) for x > 0, 


is not differentiable at x = 0. Consider the limiting process for both Ax > 0 and 
Ax <0. 

Qty, Find dy/dx if x = (t —2)/(t+ 2) and y = 2t/(t + 1) for —oo < t < ~. Show that 
it is always non-negative, and make use of this result in sketching the curve of y 
as a function of x. 

2.8 If 2y+siny + 5 = x4 + 4x3 + 22, show that dy/dx = 16 when x = 1. 

2.9 Find the second derivative of y(x) = cos[(z/2) — ax]. Now set a = 1 and verify 
that the result is the same as that obtained by first setting a = 1 and simplifying 
y(x) before differentiating. 
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2.10 


2.11 


2.12 


2.13 
2.14 


2.15 


2.16 


2.17 


2.18 


2.19 


The function y(x) is defined by y(x) = (14+ x)". 


(a) Use the chain rule to show that the first derivative of y is nmx"—!(1+x")""!. 


(b) The binomial expansion (see section 1.5) of (1 +z)" is 


nn—1) og Mn 1) (nr +1) 


Zope, 
! r! as 


(4+2)"=1+nz+ 


Keeping only the terms of zeroth and first order in dx, apply this result twice 
to derive result (a) from first principles. 

(c) Expand y in a series of powers of x before differentiating term by term. 
Show that the result is the series obtained by expanding the answer given 
for dy/dx in (a). 


Show by differentiation and substitution that the differential equation 


@y _, dy 

Lae Any ree 2 _ 

4x 2 Axe + (4x° + 3)y =0 

has a solution of the form y(x) = x" sin x, and find the value of n. 

Find the positions and natures of the stationary points of the following functions: 


(a) x3 —3x +3; (b) x3 — 3x? + 3x; (c) x74 3x43; 
(d) sinax with a 4 0; (e) x° +.x°; (f) x* — x3. 


Show that the lowest value taken by the function 3x* + 4x3 — 12x? + 6 is —26. 
By finding their stationary points and examining their general forms, determine 
the range of values that each of the following functions y(x) can take. In each 
case make a sketch-graph incorporating the features you have identified. 


(a) y(x) = (x — 1)/(x* +. 2x + 6). 
(b) y(x) = 1/(4 + 3x — x’). 
(c) y(x) = (8 sin x)/(15 + 8 tan? x). 


Show that y(x) = xa’* exp x? has no stationary points other than x = 0, if 


exp(—,/2) < a < exp(y2). 


The curve 4y? = a?(x + 3y) can be parameterised as x = acos 30, y = acos0. 


(a) Obtain expressions for dy/dx (i) by implicit differentiation and (ii) in param- 
eterised form. Verify that they are equivalent. 

(b) Show that the only point of inflection occurs at the origin. Is it a stationary 
point of inflection? 

(c) Use the information gained in (a) and (b) to sketch the curve, paying 
particular attention to its shape near the points (—a,a/2) and (a,—a/2) and 
to its slope at the ‘end points’ (a,a) and (—a,—a). 


The parametric equations for the motion of a charged particle released from rest 
in electric and magnetic fields at right angles to each other take the forms 


x = a(@ — sin 8), y =a(1—cos 9). 


Show that the tangent to the curve has slope cot(0/2). Use this result at a few 
calculated values of x and y to sketch the form of the particle’s trajectory. 

Show that the maximum curvature on the catenary y(x) = acosh(x/a) is 1/a. You 
will need some of the results about hyperbolic functions stated in subsection 3.7.6. 
The curve whose equation is x?/? + y?/3 = a’? for positive x and y and which 
is completed by its symmetric reflections in both axes is known as an astroid. 
Sketch it and show that its radius of curvature in the first quadrant is 3(axy)!/, 


77 


PRELIMINARY CALCULUS 


2.20 


2.21 


2322 


2.23 


2.24 


225 


Figure 2.13 The coordinate system described in exercise 2.20. 


A two-dimensional coordinate system useful for orbit problems is the tangential- 
polar coordinate system (figure 2.13). In this system a curve is defined by r, the 
distance from a fixed point O to a general point P of the curve, and p, the 
perpendicular distance from O to the tangent to the curve at P. By proceeding 
as indicated below, show that the radius of curvature, p, at P can be written in 
the form p = rdr/dp. 

Consider two neighbouring points, P and Q, on the curve. The normals to the 
curve through those points meet at C, with (in the limit Q — P) CP =CQ =p. 
Apply the cosine rule to triangles OPC and OQC to obtain two expressions for 
c?, one in terms of r and p and the other in terms of r+ Ar and p+ Ap. By 
equating them and letting Q — P deduce the stated result. 

Use Leibnitz’ theorem to find 


(a) the second derivative of cos x sin 2x, 
(b) the third derivative of sin x In x, 
(c) the fourth derivative of (2x* + 3x? + x + 2) exp 2x. 


If y = exp(—x?), show that dy/dx = —2xy and hence, by applying Leibnitz’ 
theorem, prove that for n> 1 


ytd + 2xy") + Any") =0. 
Use the properties of functions at their turning points to do the following: 


(a) By considering its properties near x = 1, show that f(x) = S5x*— 11x? + 
26x? — 44x + 24 takes negative values for some range of x. 

(b) Show that f(x) = tan x — x cannot be negative for 0 < x < 2/2, and deduce 
that g(x) = x7! sin x decreases monotonically in the same range. 


Determine what can be learned from applying Rolle’s theorem to the following 
functions f(x): (a) e*%; (b) x? + 6x; (c) 2x? + 3x +1; (d) 2x? + 3x +2; (e) 
2x3 — 21x? + 60x +k. (f) If k = —45 in (e), show that x = 3 is one root of 
f(x) = 0, find the other roots, and verify that the conclusions from (e) are 
satisfied. 
By applying Rolle’s theorem to x” sinnx, where n is an arbitrary positive integer, 
show that tannx +x = 0 has a solution a, with 0 < «, < a/n. Apply the 
theorem a second time to obtain the nonsensical result that there is a real « in 
0 < a <2/n, such that cos?(na) = —n. Explain why this incorrect result arises. 
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2.26 


2.27 


2.28 


2.29 
2.30 


2.31 


2.32 


2.33 


2.34 


2.35 


2.36 


2.37 


Use the mean value theorem to establish bounds in the following cases. 


(a) For —In(1— y), by considering Inx in the range O<1l—-y<x<l. 
(b) For e” — 1, by considering e* — 1 in the range 0 <x < y. 


For the function y(x) = x? exp(—x) obtain a simple relationship between y and 
dy/dx and then, by applying Leibnitz’ theorem, prove that 


xyer) + (n +x- 2)y ae ny") =0. 


Use Rolle’s theorem to deduce that, if the equation f(x) = 0 has a repeated root 
x,, then x; is also a root of the equation f’(x) = 0. 


(a) Apply this result to the ‘standard’ quadratic equation ax? + bx + c = 0, to 
show that a necessary condition for equal roots is b? = 4ac. 

(b) Find all the roots of f(x) = x? + 4x? — 3x — 18 = 0, given that one of them 
is a repeated root. 

(c) The equation f(x) = x++4x3 + 7x? + 6x + 2 = 0 has a repeated integer root. 
How many real roots does it have altogether? 


Show that the curve x? + y? — 12x — 8y — 16 = 0 touches the x-axis. 
Find the following indefinite integrals: 


(a) f(44+x?)-! dx; (b) [(8+2x—x) 1? dx for 2<x<4; 

(c) f(+sin0)' dd; (d) f(xyT—x)y'dx for 0<x<1. 

Find the indefinite integrals J of the following ratios of polynomials: 
(a) (x+3)/Q2+x—2); 

(b) (x? +. 5x? + 8x + 12)/(2x? + 10x + 12); 


(c) (3x? + 20x + 28)/(x? + 6x + 9); 
(d) x3/(a8 + x8). 


Express x?(ax +b) 
hence evaluate 


—! as the sum of powers of x and another integrable term, and 


b/a x2 
= dx. 
[ ax +b * 


Find the integral J of (ax? + bx + c)~!, with a # 0, distinguishing between the 
cases (i) b* > 4ac, (ii) b? < 4ac and (iii) b? = 4ac. 
Use logarithmic integration to find the indefinite integrals J of the following: 


(a) sin2x/(1 + 4sin’ x); 

(b) e*/(e*—e™); 

(c) (1+ xInx)/(xInx); 

(d) [x(x" + a")]7. 

Find the derivative of f(x) = (1+sinx)/cosx and hence determine the indefinite 
integral J of sec x. 

Find the indefinite integrals, J, of the following functions involving sinusoids: 
(a) cos> x —cos* x; 

(b) (1 —cos x)/(1 + cos x); 


(c) cosxsinx/(1+cosx); 
(d) sec? x/(1 — tan? x). 


By making the substitution x = acos* 6 + bsin’ 6, evaluate the definite integrals 
J between limits a and b (> a) of the following functions: 


(a) [(x-a)\(b—x)]-?; 
(b) [(x-a)(b—x)]'"; 
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2.38 


2.39 


2.40 


2.41 


2.42 


2.43 


2.44 


(c) [(x—a)/(b— x)”. 


Determine whether the following integrals exist and, where they do, evaluate 
them: 


co 00 x 

(a) exp(—Ax)dx;  (b) dx; 
a in C +@yP 

© f qe @ | Sx; 


n/2 1 x 
© | cot 0d0; o | awn 


Use integration by parts to evaluate the following: 


y y 
(a) f x’? sinxdx;  (b) / xInx dx; 
0 1 


y y 
(c) i sin'xdx;  (d) i In(a? + x”)/x? dx. 
0 1 
Show, using the following methods, that the indefinite integral of x3/(x + 1)'/? is 


J = 2(5x° — 6x? + 8x — 16)(x + 1)! +e. 


(a) Repeated integration by parts. 
(b) Setting x + 1 =u? and determining dJ/du as (dJ/dx)(dx/du). 


The gamma function I’(n) is defined for all n > —1 by 
T(n+ 1) = | x"e~* dx. 
0 


Find a recurrence relation connecting '(n + 1) and I'(n). 


(a) Deduce (i) the value of [(n+ 1) when n is a non-negative integer, and (ii) 
the value of I (4), given that '($) = J. 
(b) Now, taking factorial m for any m to be defined by m! = I'(m-+ 1), evaluate 


Define J(m,n), for non-negative integers m and n, by the integral 
n/2 
J(m,n) = Yi cos” @ sin" 0 dd. 
0 


(a) Evaluate J(0,0), J(0, 1), J(1,0), J(1, 1), J(m, 1), JU, n). 

(b) Using integration by parts, prove that, for m and n both > 1, 
-1 
+n 


J(m,n) = ~ J(m—2,n) and J(m,n) = = Jom, n—2). 


(c) Evaluate (i) J(5, 3), (ii) J(6, 5) and (iii) J(4, 8). 


By integrating by parts twice, prove that I, as defined in the first equality below 
for positive integers n has the value given in the second equality: 


n—sin(nn/2) 


n/2 
i= f sinn0 cos@ dd = 
0 n—1 


Evaluate the following definite integrals: 
(a) Io. xe* dx;  (b) f [08 +1)/(xt+4x + 1] dx; 


(c) [? [a+(a—1)cos@]-'d0 witha>4; (d) [% (x? + 6x + 18)! dx. 
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2.45 


2.46 


2.47 


2.48 


2.49 


2.50 


2.1 
2.3 


2:5 


2.7 


2.9 
2.11 


2.13 


2.15 
2.17 


2.19 


If J, is the integral 
[ x" exp(—x?) dx 
0 
show that 


(a) Jars = (r!)/2, 
b) Jy, = 2-7(2r — 1)(2r — 3)- +» -(5)(3)(1) Jo. 


Find positive constants a, b such that ax < sinx < bx for 0 < x < 2/2. Use 


this inequality to find (to two significant figures) upper and lower bounds for the 
integral 


n/2 
if -| (1+ sin x)!/? dx. 
0 


Use the substitution t = tan(x/2) to evaluate I exactly. 
By noting that for 0 <7 < 1, y'/? > 73/4 > y, prove that 


2 1 “4 23/4 us 
ssanfe — x") dx <q. 


Show that the total length of the astroid x7/? + y?/? = a?/?, which can be 
parameterised as x = acos? 0, y = asin’ 0, is 6a. 

By noting that sinhx < $e* < coshx, and that 1 +2? < (1+) for z > 0, show 
that, for x > 0, the length L of the curve y = ze measured from the origin 
satisfies the inequalities sinhx < L <x+sinhx. 

The equation of a cardioid in plane polar coordinates is 


p=a(1—sin@). 
Sketch the curve and find (1) its area, (ii) its total length, (iii) the surface area of 


the solid formed by rotating the cardioid about its axis of symmetry and (iv) the 
volume of the same solid. 


2.4 Hints and answers 


(a) 3; (b) 2x + 1, 2, 0; (c) cos x. 

Use: the product rule in (a), (b), (d) and (e)[3 factors]; the chain rule in (c), (f) 
and (g); logarithmic differentiation in (g) and (h). 

(a) (x? + 2x) exp x; (b) 2(cos? x — sin? x) = 2. cos 2x; 

(c) 2cos 2x; (d) sinax + ax cos ax; 

(e) (aexp ax)[(sin ax + cos ax) tan“! ax + (sinax)(1 + a?x?)7']; 

(f) fax — x~*)]/Ex(" + x]; (g) [a — a*) Ina] /(a + a); (h) (1 + In x)". 
(a) —6(2x + 3)~*; (b) 2 sec” x tan x; (c) —9 cosech?3x coth 3x; 

(d) —x"(In x); (€) (a? — x2)" fsin“"(x/a)], 

Calculate dy/dt and dx/dt and divide one by the other. (¢ + 2)?/[2(t + 1)]. 
Alternatively, eliminate t and find dy/dx by implicit differentiation. 

—sin x in both cases. 

The required conditions are 8n — 4 = 0 and 4n? — 8n + 3 = 0; both are satisfied 
by n= 35. 

The stationary points are the zeros of 12x? + 12x? — 24x. The lowest stationary 
value is —26 at x = —2; other stationary values are 6 at x =O and 1 atx =1. 
Use logarithmic differentiation. Set dy/dx = 0, obtaining 2x? + 2x Ina+1=0. 
See figure 2.14. 

dy y\'B dy a 
BeOS) 2 ge ee 


2/3 
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2.21 


2.23 


2.25 


2.27 


2.29 


2.31 


2.33 


2.35 
2.37 


2.39 


2.41 
2.43 
2.45 
2.47 


2.49 


2a 


Ta 2na 


Figure 2.14 The solution to exercise 2.17. 


(a) 2(2 — 9 cos? x) sin x; (b) (2x3 — 3x7!) sin x — (3x7? + In x) cos x; (c) 8(4x3 + 

30x? + 62x + 38) exp 2x. 

(a) f(1) =0 whilst f’(1) £0, and so f(x) must be negative in some region with 
x = 1 as an endpoint. 

(b) f’(x) = tan?x > 0 and f(0) = 0; g(x) = (—cosx)(tan x — x)/x’, which is 
never positive in the range. 

The false result arises because tannx is not differentiable at x = 2/(2n), which 

lies in the range 0 < x < z/n, and so the conditions for applying Rolle’s theorem 

are not satisfied. 

The relationship is x dy/dx = (2—x)y. 

By implicit differentiation, y'(x) = (3x? — 12)/(8 — 3y’), giving y'(+2) = 0. Since 

y(2) =4 and y(—2) = 0, the curve touches the x-axis at the point (—2,0). 

(a) Express in partial fractions; J = + In[(x —1)4/(x+2)J +c. 

(b) Divide the numerator by the denominator and express the remainder in 
partial fractions; J = x7/4 + 4In(x + 2) — 3In(x + 3) +c. 

(c) After division of the numerator by the denominator, the remainder can be 
expressed as 2(x + 3)~! — 5(x +. 3)-?; J = 3x + 2In(x + 3) 4 5(x +3)! +0. 

(d) Set xt =u; J = (4a*)“! tan“! (x4/a*) +c. 

Writing b? — 4ac as A? > 0, or 4ac — b* as A? > 0: 

(i) AT! In[(2ax + b — A)/(2ax + b + A)] +k; 

(ii) 2A’ tan [(2ax + b)/A'] +k; 

(iii) —2(2ax + b)y-! +k. 

f'(x) = (1 + sin x)/ cos? x = f(x) sec x; J = In(f(x)) +c = In(sec x + tanx) +c. 

Note that dx = 2(b —a)cos 0 sin @ dé. 

(a) m; (b) m(b—a)?/8; (c) x(b — a)/2. 

(a) (2—y?)cosy + 2ysiny — 2; (b) [(y?Iny)/2] + [1 —y*)/4]; 

(c) ysin™ y + (1—y?)'? — 1; 

(d) In(a* + 1) — (1/y)In(a@? + y’) + (2/a)[tan7!(y/a) — tan7!(1/a)]. 

T(n+ 1) = nF (n); (a) (i) n}, (ii) 15./2/8; (b) —2,/a. 

By integrating twice, recover a multiple of I,,. 

Jor41 => rJ>)—1 and QJ. => (2r = 1)Jo)~2. 

Set 7 = 1 —(x/a)? throughout, and x = asin0@ in one of the bounds. 


L=fp (1+ 1 exp 2x)!” dx. 
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Complex numbers and 
hyperbolic functions 


This chapter is concerned with the representation and manipulation of complex 
numbers. Complex numbers pervade this book, underscoring their wide appli- 
cation in the mathematics of the physical sciences. The application of complex 
numbers to the description of physical systems is left until later chapters and 
only the basic tools are presented here. 


3.1 The need for complex numbers 


Although complex numbers occur in many branches of mathematics, they arise 
most directly out of solving polynomial equations. We examine a specific quadratic 
equation as an example. 

Consider the quadratic equation 


7? —424+5=0. (3.1) 
Equation (3.1) has two solutions, z; and z, such that 
(z —21)\(z — 22) = 0. (3.2) 


Using the familiar formula for the roots of a quadratic equation, (1.4), the 
solutions z; and z9, written in brief as z;2, are 


44 JP 40 x5) 
=e" 
=2+ v4 (3.3) 


2 
Both solutions contain the square root of a negative number. However, it is not 
true to say that there are no solutions to the quadratic equation. The fundamental 
theorem of algebra states that a quadratic equation will always have two solutions 
and these are in fact given by (3.3). The second term on the RHS of (3.3) is 
called an imaginary term since it contains the square root of a negative number; 


71,2 
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i 3 3 ie 
Figure 3.1 The function f(z) = z*—4z +5. 


the first term is called a real term. The full solution is the sum of a real term 
and an imaginary term and is called a complex number. A plot of the function 
f(z) = 2? —4z +5 is shown in figure 3.1. It will be seen that the plot does not 
intersect the z-axis, corresponding to the fact that the equation f(z) = 0 has no 
purely real solutions. 

The choice of the symbol z for the quadratic variable was not arbitrary; the 
conventional representation of a complex number is z, where z is the sum of a 
real part x and i times an imaginary part y, Le. 


z=x-+iy, 


where i is used to denote the square root of —1. The real part x and the imaginary 
part y are usually denoted by Rez and Imz respectively. We note at this point 
that some physical scientists, engineers in particular, use j instead of i. However, 
for consistency, we will use i throughout this book. 
In our particular example, ./—4 = 2,/—I = 2i, and hence the two solutions of 
(3.1) are 
2i 


mg =245 =24i. 


Thus, here x = 2 and y= +1. 
For compactness a complex number is sometimes written in the form 
z = (x,y), 


where the components of z may be thought of as coordinates in an xy-plot. Such 
a plot is called an Argand diagram and is a common representation of complex 
numbers; an example is shown in figure 3.2. 
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Figure 3.2 The Argand diagram. 


Our particular example of a quadratic equation may be generalised readily to 
polynomials whose highest power (degree) is greater than 2, e.g. cubic equations 
(degree 3), quartic equations (degree 4) and so on. For a general polynomial f(z), 
of degree n, the fundamental theorem of algebra states that the equation f(z) = 0 
will have exactly n solutions. We will examine cases of higher-degree equations 
in subsection 3.4.3. 

The remainder of this chapter deals with: the algebra and manipulation of 
complex numbers; their polar representation, which has advantages in many 
circumstances; complex exponentials and logarithms; the use of complex numbers 
in finding the roots of polynomial equations; and hyperbolic functions. 


3.2. Manipulation of complex numbers 


This section considers basic complex number manipulation. Some analogy may 
be drawn with vector manipulation (see chapter 7) but this section stands alone 
as an introduction. 


3.2.1 Addition and subtraction 


The addition of two complex numbers, z; and zz, in general gives another 
complex number. The real components and the imaginary components are added 
separately and in a like manner to the familiar addition of real numbers: 


Zp +22 = (x1 + iy) + (2 + ivr) = (X1 + X2) + 1 + ya), 
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Imz 


Figure 3.3. The addition of two complex numbers. 


or in component notation 


21 +22 = (x1, V1) + (X2, ya) = (X1 + X2, 1 + V2). 


The Argand representation of the addition of two complex numbers is shown in 
figure 3.3. 

By straightforward application of the commutativity and associativity of the 
real and imaginary parts separately, we can show that the addition of complex 
numbers is itself commutative and associative, i.e. 


714+22=224+ 21, 
21 + (z2 +23) = (21 +22) +23. 


Thus it is immaterial in what order complex numbers are added. 


> Sum the complex numbers 1 + 2i, 3— 4i, —2 +i. 


Summing the real terms we obtain 


14+3-2=2, 
and summing the imaginary terms we obtain 
2i—4i +i = —i. 


Hence 


(1+2i/) +G—4i) +(-2+i =2-i< 


The subtraction of complex numbers is very similar to their addition. As in the 
case of real numbers, if two identical complex numbers are subtracted then the 
result is zero. 
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Imz 


yb-------3 


Figure 3.4 The modulus and argument of a complex number. 


3.2.2 Modulus and argument 


The modulus of the complex number z is denoted by |z| and is defined as 


|z| = x2 4+ y?. (3.4) 


Hence the modulus of the complex number is the distance of the corresponding 
point from the origin in the Argand diagram, as may be seen in figure 3.4. 
The argument of the complex number z is denoted by arg z and is defined as 


arg z = tan! (=) . (3.5) 
x 

It can be seen that argz is the angle that the line joining the origin to z on 
the Argand diagram makes with the positive x-axis. The anticlockwise direction 
is taken to be positive by convention. The angle arg z is shown in figure 3.4. 
Account must be taken of the signs of x and y individually in determining in 
which quadrant arg z lies. Thus, for example, if x and y are both negative then 
arg z lies in the range —z < arg z < —z/2 rather than in the first quadrant 
(0 < arg z < 2/2), though both cases give the same value for the ratio of y to x. 


> Find the modulus and the argument of the complex number z = 2 — 3i. 


Using (3.4), the modulus is given by 


|z| = /2? + (3 = VI3. 


Using (3.5), the argument is given by 


arg z = tan”! (—3). 


The two angles whose tangents equal —1.5 are —0.9828 rad and 2.1588 rad. Since x = 2 and 
y =—3, z clearly lies in the fourth quadrant; therefore arg z = —0.9828 is the appropriate 
answer. < 
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3.2.3 Multiplication 


Complex numbers may be multiplied together and in general give a complex 
number as the result. The product of two complex numbers z; and z, is found 
by multiplying them out in full and remembering that i? = —1, ice. 


2122 = (x1 + iy1)(x2 + iva) 
= x1x2 + ix1y2 + 1x2 +7 yr 
= (X1X2 — y1y2) + i(x1y2 + y1X2). (3.6) 


> Multiply the complex numbers z; = 3 + 2i and z2 = —1 —4i. 


By direct multiplication we find 
2122 = (3 + 2i)(—1 — 4i) 
= —3—2i- 121-87 
=5-14i.< (3.7) 


The multiplication of complex numbers is both commutative and associative, 
Le. 
2120 = 2221, (3.8) 
(2122)23 = 21 (2223). (3.9) 
The product of two complex numbers also has the simple properties 
|Z1Z2| = |z1||Z2|, (3.10) 
arg(z1Z2) = arg z; + arg Zo. (3.11) 


These relations are derived in subsection 3.3.1. 


> Verify that (3.10) holds for the product of z; = 3+ 2i and z, = —1 —4i. 


From (3.7) 
|z122| = |5— 14i] = 9 + (—14P = 221. 
We also find 
la] = V3?+2? = VI, 
Iza] = V/(-1? + (4? = V7, 
and hence 


\zil|z2] = 1317 = 221 = |z12|. < 


We now examine the effect on a complex number z of multiplying it by +1 
and +i. These four multipliers have modulus unity and we can see immediately 
from (3.10) that multiplying z by another complex number of unit modulus gives 
a product with the same modulus as z. We can also see from (3.11) that if we 
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Imz 


Figure 3.5 Multiplication of a complex number by +1 and +i. 


multiply z by a complex number then the argument of the product is the sum 
of the argument of z and the argument of the multiplier. Hence multiplying 
z by unity (which has argument zero) leaves z unchanged in both modulus 
and argument, ie. z is completely unaltered by the operation. Multiplying by 
—1 (which has argument z) leads to rotation, through an angle z, of the line 
joining the origin to z in the Argand diagram. Similarly, multiplication by i or —i 
leads to corresponding rotations of 2/2 or —7/2 respectively. This geometrical 
interpretation of multiplication is shown in figure 3.5. 


> Using the geometrical interpretation of multiplication by i, find the product i(1 — i). 


The complex number 1 —i has argument —1z/4 and modulus ./2. Thus, using (3.10) and 
(3.11), its product with i has argument +7/4 and unchanged modulus ./2. The complex 
number with modulus ./2 and argument +7/4 is 1 +i and so 

i(1—-ij) =1+i, 


as is easily verified by direct multiplication. < 


The division of two complex numbers is similar to their multiplication but 
requires the notion of the complex conjugate (see the following subsection) and 
so discussion is postponed until subsection 3.2.5. 


3.2.4 Complex conjugate 


If z has the convenient form x + iy then the complex conjugate, denoted by z’*, 
may be found simply by changing the sign of the imaginary part, 1e. if z = x +iy 
then z* = x —iy. More generally, we may define the complex conjugate of z as 
the (complex) number having the same magnitude as z that when multiplied by 
z leaves a real result, i.e. there is no imaginary component in the product. 
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Figure 3.6 The complex conjugate as a mirror image in the real axis. 
In the case where z can be written in the form x + iy it is easily verified, by 
direct multiplication of the components, that the product zz” gives a real result: 
zz” =(x +iy)\(x —iy) =x* —ixytixy —Py? =x? + y*? =[z/. 


Complex conjugation corresponds to a reflection of z in the real axis of the 
Argand diagram, as may be seen in figure 3.6. 


> Find the complex conjugate of z = a+ 2i-+ 3ib. 


The complex number is written in the standard form 


z=a+i(2+ 3b); 
then, replacing i by —i, we obtain 


z* =a—i(2+3b). < 


In some cases, however, it may not be simple to rearrange the expression for 
z into the standard form x + iy. Nevertheless, given two complex numbers, 2; 
and zp», it is straightforward to show that the complex conjugate of their sum 
(or difference) is equal to the sum (or difference) of their complex conjugates, 1.e. 
(z; +22)" =z; +25. Similarly, it may be shown that the complex conjugate of the 
product (or quotient) of z; and z) is equal to the product (or quotient) of their 
complex conjugates, Le. (z}22)" = 2z;25 and (z1/z2)" = z;/z}. 

Using these results, it can be deduced that, no matter how complicated the 
expression, its complex conjugate may always be found by replacing every i by 
—i. To apply this rule, however, we must always ensure that all complex parts are 
first written out in full, so that no i’s are hidden. 
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> Find the complex conjugate of the complex number z = w°*?), where w = x + Si. 


Although we do not discuss complex powers until section 3.5, the simple rule given above 
still enables us to find the complex conjugate of z. 

In this case w itself contains real and imaginary components and so must be written 
out in full, Le. 


z= wor tix = (x + Sippy tee, 
Now we can replace each i by —i to obtain 
z* = (x — SiO), 


It can be shown that the product zz” is real, as required. < 


The following properties of the complex conjugate are easily proved and others 
may be derived from them. If z = x + iy then 


(z")" =z, (3.12) 
z+z" =2 Rez = 2x, (3.13) 
z—z =2iImz = Diy, (3.14) 

Zz x? — y? : 2xy 
a a (=>) +i (=) . (3.15) 


The derivation of this last relation relies on the results of the following subsection. 


3.2.5 Division 


The division of two complex numbers z; and z2 bears some similarity to their 
multiplication. Writing the quotient in component form we obtain 

Zz xi +i 

ue Eats 2 (3.16) 

22 -X2 + iy2 
In order to separate the real and imaginary components of the quotient, we 
multiply both numerator and denominator by the complex conjugate of the 
denominator. By definition, this process will leave the denominator as a real 
quantity. Equation (3.16) gives 


Z1 (x1 tiyi)(x2 — iy2) — (1x2 + 1 y2) + ie2y1 — x1y2) 


22 (Xz + iy2)(x2 — iy2) x3 + y3 
_ XixaT yiy2 | .X2V1 ~X1Y2 
x3 +3 x3 + y3 


Hence we have separated the quotient into real and imaginary components, as 
required. 

In the special case where z) = 2}, so that x. = x; and yy = —y, the general 
result reduces to (3.15). 
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> Express z in the form x +iy, when 


Multiplying numerator and denominator by the complex conjugate of the denominator 
we obtain 


(3—2i(-1—4i) _ —11—10i 


ee eae ee an 
11 10. 
7 17 


In analogy to (3.10) and (3.11), which describe the multiplication of two 
complex numbers, the following relations apply to division: 


A) leat (3.17) 
Z2| (za 
arg (2) = arg Zz; — arg Z. (3.18) 


The proof of these relations is left until subsection 3.3.1. 


3.3 Polar representation of complex numbers 


Although considering a complex number as the sum of a real and an imaginary 
part is often useful, sometimes the polar representation proves easier to manipulate. 
This makes use of the complex exponential function, which is defined by 


2 3 


see eee (3.19) 


ef =expz=1l+z+5, 31 


Strictly speaking it is the function expz that is defined by (3.19). The number e 
is the value of exp(1), ie. it is just a number. However, it may be shown that e” 
and expz are equivalent when z is real and rational and mathematicians then 
define their equivalence for irrational and complex z. For the purposes of this 
book we will not concern ourselves further with this mathematical nicety but, 
rather, assume that (3.19) is valid for all z. We also note that, using (3.19), by 
multiplying together the appropriate series we may show that (see chapter 24) 


a (3.20) 
which is analogous to the familiar result for exponentials of real numbers. 
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Imz 


Figure 3.7 The polar representation of a complex number. 


From (3.19), it immediately follows that for z = i0, @ real, 


io 
iO __ 1 7, ae ea 
1+i0 a art (3.21) 
6264 : ae 6 
and hence that 
e” = cos +isin 0, (3.23) 


where the last equality follows from the series expansions of the sine and cosine 
functions (see subsection 4.6.3). This last relationship is called Euler’s equation. It 
also follows from (3.23) that 


e”? = cosnO +isinnd 


for all n. From Euler’s equation (3.23) and figure 3.7 we deduce that 
re” = r(cos0 + isin 0) 
=xX+iy. 
Thus a complex number may be represented in the polar form 
z=re”. (3.24) 


Referring again to figure 3.7, we can identify r with |z| and @ with arg z. The 
simplicity of the representation of the modulus and argument is one of the main 
reasons for using the polar representation. The angle 0 lies conventionally in the 
range —z < 0 < a, but, since rotation by @ is the same as rotation by 2nz + 0, 
where n is any integer, 


0 0+2nz) 


re” = rell : 
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Imz 


ryrpelll+02) 


Figure 3.8 The multiplication of two complex numbers. In this case r; and 
ry are both greater than unity. 


The algebra of the polar representation is different from that of the real and 
imaginary component representation, though, of course, the results are identical. 
Some operations prove much easier in the polar representation, others much more 
complicated. The best representation for a particular problem must be determined 
by the manipulation required. 


3.3.1 Multiplication and division in polar form 


Multiplication and division in polar form are particularly simple. The product of 


z1 =ryjel™ and z. = rye"? is given by 


z122 = rere” 


= ry ret), (3.25) 


The relations |z122| = |z1||z2| and arg(ziz2) = arg z; + arg z2 follow immediately. 
An example of the multiplication of two complex numbers is shown in figure 3.8. 


Division is equally simple in polar form; the quotient of z; and z2 is given by 


id 

Zz rye! Tie 

Ss = = eli), (3.26) 
22 rel? r2 


The relations |z,;/z2| = |z,|/|z2| and arg(z,;/z.) = arg z; — arg z) are again 
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Figure 3.9 The division of two complex numbers. As in the previous figure, 
r; and r2 are both greater than unity. 


immediately apparent. The division of two complex numbers in polar form is 
shown in figure 3.9. 


3.4 de Moivre’s theorem 


. . : i n i; 
We now derive an extremely important theorem. Since (e’”) = el"®, we have 


(cos 0 + isin 0)" = cosné + isin nO, (3.27) 


where the identity e”? = cosn0 +isinn@ follows from the series definition of 


e”9 (see (3.21)). This result is called de Moivre’s theorem and is often used in the 
manipulation of complex numbers. The theorem is valid for all n whether real, 
imaginary or complex. 

There are numerous applications of de Moivre’s theorem but this section 
examines just three: proofs of trigonometric identities; finding the nth roots of 
unity; and solving polynomial equations with complex roots. 


3.4.1 Trigonometric identities 


The use of de Moivre’s theorem in finding trigonometric identities is best illus- 
trated by example. We consider the expression of a multiple-angle function in 
terms of a polynomial in the single-angle function, and its converse. 
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> Express sin30 and cos 30 in terms of powers of cos @ and sin 0. 


Using de Moivre’s theorem, 


cos 30 + isin 30 = (cos 0 + isin 0)> 
= (cos? 0 — 3cos 0 sin’ 0) + i(3 sin 0 cos? 0 — sin’ 0). (3.28) 


We can equate the real and imaginary coefficients separately, i.e. 


cos 30 = cos* 0 — 3cos 0 sin? 0 
= 4cos* 6 — 3cos0 (3.29) 


and 


sin 30 = 3 sin @ cos’ 6 — sin’ 0 
=3sin0—4sin’ 0. < 


This method can clearly be applied to finding power expansions of cos n@ and 
sinn@ for any positive integer n. 


The converse process uses the following properties of z = e”, 
£4 ll 
z" + — =2cosné, (3.30) 
gn 
zt ke = 2isinnd. (3.31) 
VAL 


These equalities follow from simple applications of de Moivre’s theorem, i.e. 


ze - = (cos @ + isin 0)" + (cos@ +isin0)” 
= cosné + isinnd + cos(—né) + isin(—n0) 
= cosnd + isinnd + cosné —isinnd 
= 2cosn0 
and 
gi + = (cos @ + isin 0)" — (cos? +isin0)”" 


=cosn0 + isinn# —cosn@ + isinnd 


= 2isinné. 


In the particular case where n = 1, 


z+ A = 2 + 6 — 2c0s80, (3.32) 
ia 
1 ‘ 

z—-=e% — 6 — Disind. (3.33) 
Zz 
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> Find an expression for cos* 0 in terms of cos 30 and cos 0. 


Using (3.32), 
1 ie 
Fi ate 
iy gee 3. 1 
3 (< +3z+ a + =) 


1 aa TN 3 ie 1 
8 za) 8 z)° 
Now using (3.30) and (3.32), we find 


cos’ 9 = 4.c0s 30 + 3 cos 0. < 


cos? 0 = 


This result happens to be a simple rearrangement of (3.29), but cases involving 
larger values of n are better handled using this direct method than by rearranging 
polynomial expansions of multiple-angle functions. 


3.4.2 Finding the nth roots of unity 


The equation z? = 1 has the familiar solutions z = +1. However, now that 
we have introduced the concept of complex numbers we can solve the general 
equation z” = 1. Recalling the fundamental theorem of algebra, we know that 
the equation has n solutions. In order to proceed we rewrite the equation as 


Zt = ezikn 


? 


where k is any integer. Now taking the nth root of each side of the equation we 
find 


7 = eriknin. 
Hence, the solutions of z" = 1 are 
Zan = 1, etn e2iln—t)n/n, 
corresponding to the values 0,1,2,...,n —1 for k. Larger integer values of k do 


not give new solutions, since the roots already listed are simply cyclically repeated 
fork =n,n+1,n+2, etc. 


> Find the solutions to the equation z> = 1. 


By applying the above method we find 


z= evikn/3. 


Hence the three solutions are z; = e% = 1, z) = e?"/3, z, . We note that, as expected, 


the next solution, for which k = 3, gives z4 = e%*/3 = 1 =z), so that there are only three 
separate solutions. < 


efin/3 
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Figure 3.10 The solutions of z? = 1. 


Not surprisingly, given that |z*| = |z|> from (3.10), all the roots of unity have 
unit modulus, ie. they all lie on a circle in the Argand diagram of unit radius. 
The three roots are shown in figure 3.10. 

The cube roots of unity are often written 1, m and w?. The properties w* = 1 
and 1+@+* =0 are easily proved. 


3.4.3 Solving polynomial equations 


A third application of de Moivre’s theorem is to the solution of polynomial 
equations. Complex equations in the form of a polynomial relationship must first 
be solved for z in a similar fashion to the method for finding the roots of real 
polynomial equations. Then the complex roots of z may be found. 


> Solve the equation 2° — z> + 4z4 — 623 + 227 —8z2+8=0. 


We first factorise to give 


(23 — 2)(z? + 4)(z — 1) =0. 


Hence z* = 2 or z? 4 or z = 1. The solutions to the quadratic equation are z = +2i; 
to find the complex cube roots, we first write the equation in the form 


2 Se 2e7r 
where k is any integer. If we now take the cube root, we get 


z= 2/3 g2ikn/3. 


98 


3.5 COMPLEX LOGARITHMS AND COMPLEX POWERS 


To avoid the duplication of solutions, we use the fact that —z < arg z < m and find 


414= PER 
F 1 3 
zy = UB e2A/3 = 91/3 ( 5 ap $1) ; 
: 1 3 
m= 21/3 9-2ni/3 _ 91/3 (-} _ ¥) : 
The complex numbers z,, z. and 23, together with z4 = 2i, z; = —2i and zs = 1 are the 


solutions to the original polynomial equation. 
As expected from the fundamental theorem of algebra, we find that the total number 
of complex roots (six, in this case) is equal to the largest power of z in the polynomial. < 


A useful result is that the roots of a polynomial with real coefficients occur in 
conjugate pairs (i.e. if z; is a root, then z; is a second distinct root, unless z; is 
real). This may be proved as follows. Let the polynomial equation of which z is 
a root be 


yz" + yz" | +++ Faz tay =0. 
Taking the complex conjugate of this equation, 
a(z") 4 at_y(z*)l +-+ tajz" +44 = 0. 


But the a, are real, and so z”* satisfies 


Ay(z")" + dn1(2")"! +++- + az" + ap =0, 


and is also a root of the original equation. 


3.5 Complex logarithms and complex powers 


The concept of a complex exponential has already been introduced in section 3.3, 
where it was assumed that the definition of an exponential as a series was valid 
for complex numbers as well as for real numbers. Similarly we can define the 
logarithm of a complex number and we can use complex numbers as exponents. 

Let us denote the natural logarithm of a complex number z by w = Lnz, where 
the notation Ln will be explained shortly. Thus, w must satisfy 


z=e”. 
Using (3.20), we see that 
212) = eMteM? = eMit™2, 
and taking logarithms of both sides we find 
Ln (2,22) = wy + wy = Lnz, + Lnz, (3.34) 


which shows that the familiar rule for the logarithm of the product of two real 
numbers also holds for complex numbers. 
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We may use (3.34) to investigate further the properties of Ln z. We have already 
noted that the argument of a complex number is multivalued, 1.e. arg z = 0+2nnz, 
where n is any integer. Thus, in polar form, the complex number z should strictly 
be written as 

i(0-+2nn) 


z=re : 
Taking the logarithm of both sides, and using (3.34), we find 
Lnz =Inr+i(@ + 2nz), (3.35) 


where Inr is the natural logarithm of the real positive quantity r and so is 
written normally. Thus from (3.35) we see that Lnz is itself multivalued. To avoid 
this multivalued behaviour it is conventional to define another function Inz, the 
principal value of Lnz, which is obtained from Lnz by restricting the argument 
of z to lie in the range —1 <0 <7. 


> Evaluate Ln (—i). 
By rewriting —i as a complex exponential, we find 
Ln (—i) = Ln [e("/4")] = i(—n/2 + 2nn), 


where n is any integer. Hence Ln(—i) = —ia/2, 3in/2, .... We note that In(—i), the 
principal value of Ln (—i), is given by In(—i) = —iz/2. < 


If z and t are both complex numbers then the zth power of t is defined by 


v= ein 


Since Lnt is multivalued, so too is this definition. 


> Simplify the expression z = i-”'. 


Firstly we take the logarithm of both sides of the equation to give 
Lnz = —2iLni. 


Now inverting the process we find 
elnz =z= eT 2iLni 


i(z/2+2nt) where n is any integer, and hence 


Lni=Ln Eee 


=i(2/2+2nn). 


We can write i = e' 


We can now simplify z to give 
j72! = e2ixi(n/2+2nz) 
_ elt t4nn) 


which, perhaps surprisingly, is a real quantity rather than a complex one. < 
Complex powers and the logarithms of complex numbers are discussed further 


in chapter 24. 
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3.6 Applications to differentiation and integration 


We can use the exponential form of a complex number together with de Moivre’s 
theorem (see section 3.4) to simplify the differentiation of trigonometric functions. 


> Find the derivative with respect to x of e** cos 4x. 


We could differentiate this function straightforwardly using the product rule (see subsec- 
tion 2.1.2). However, an alternative method in this case is to use a complex exponential. 
Let us consider the complex number 


z =e**(cos4x + isin4x) = eet = e+4ix, 


where we have used de Moivre’s theorem to rewrite the trigonometric functions as a com- 
plex exponential. This complex number has e3* cos 4x as its real part. Now, differentiating 


z with respect to x we obtain 
d. ? : : 
= = (3 + 4i)e3*** = (3 + 4i)e**(cos 4x + isin 4x), (3.36) 


where we have again used de Moivre’s theorem. Equating real parts we then find 
d : 
or (e** cos 4x) = e**(3 cos 4x — 4sin 4x). 

By equating the imaginary parts of (3.36), we also obtain, as a bonus, 


d : 
rT (e** sin 4x) = e**(4cos4x + 3 sin 4x). 

In a similar way the complex exponential can be used to evaluate integrals 
containing trigonometric and exponential functions. 


> Evaluate the integral I = { e* cos bx dx. 


Let us consider the integrand as the real part of the complex number 
e““(cos bx + isin bx) = et eibx — elatibys 


where we use de Moivre’s theorem to rewrite the trigonometric functions as a complex 
exponential. Integrating we find 


poe: yim aud +e 
a+ib 

(a — ib)elatibs 

(a — ib)(a + ib) 

ae (ae’* — ibe”) +c, (3.37) 

where the constant of integration c is in general complex. Denoting this constant by 

c= c; + ic) and equating real parts in (3.37) we obtain 


a+b? 


which agrees with result (2.37) found using integration by parts. Equating imaginary parts 
in (3.37) we obtain, as a bonus, 


I Sie cos bx dx = (acos bx + bsin bx) + 1, 


ax 


J = fe sin bx dx = (asin bx — bcos bx) +c. <4 


a + b? 
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3.7 Hyperbolic functions 


The hyperbolic functions are the complex analogues of the trigonometric functions. 
The analogy may not be immediately apparent and their definitions may appear 
at first to be somewhat arbitrary. However, careful examination of their properties 
reveals the purpose of the definitions. For instance, their close relationship with 
the trigonometric functions, both in their identities and in their calculus, means 
that many of the familiar properties of trigonometric functions can also be applied 
to the hyperbolic functions. Further, hyperbolic functions occur regularly, and so 
giving them special names is a notational convenience. 


3.7.1 Definitions 


The two fundamental hyperbolic functions are coshx and sinh x, which, as their 
names suggest, are the hyperbolic equivalents of cos x and sin x. They are defined 
by the following relations: 


cosh x = s(e* +e~*), (3.38) 
sinh x = z(e* —e~*), (3.39) 


Note that coshx is an even function and sinhx is an odd function. By analogy 
with the trigonometric functions, the remaining hyperbolic functions are 


tanh x = — (3.40) 


coshx e%+e-*’ 
1 2 
sech x = aha See (3.41) 
ori ly 2 
cosecn xX = sah x => poe’ (3.42) 
X 1 px 
Siete ee (3.43) 


tanhx e%—e~* 


All the hyperbolic functions above have been defined in terms of the real variable 
x. However, this was simply so that they may be plotted (see figures 3.11-3.13); 
the definitions are equally valid for any complex number z. 


3.7.2. Hyperbolic—trigonometric analogies 


In the previous subsections we have alluded to the analogy between trigonometric 
and hyperbolic functions. Here, we discuss the close relationship between the two 
groups of functions. 
Recalling (3.32) and (3.33) we find 
cosix = 5(e* +e~*), 


24 Ley x _ 
sin ix = zi(e* —e~). 
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cosh x 


Figure 3.11 Graphs of coshx and sechx. 


4 \ cosech x 
\ sinh x 
2 
—2 _ 1 1 2% 
V=2 
cosech x \ 


Figure 3.12 Graphs of sinh x and cosechx. 


Hence, by the definitions given in the previous subsection, 


cosh x = cos ix, (3.44) 
isinh x = sinix, (3.45) 
cos x = cosh ix, (3.46) 
isin x = sinhix. (3.47) 


These useful equations make the relationship between hyperbolic and trigono- 
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cothx 


Figure 3.13 Graphs of tanh x and cothx. 


metric functions transparent. The similarity in their calculus is discussed further 
in subsection 3.7.6. 


3.7.3 Identities of hyperbolic functions 


The analogies between trigonometric functions and hyperbolic functions having 
been established, we should not be surprised that all the trigonometric identities 
also hold for hyperbolic functions, with the following modification. Wherever 
sin? x occurs it must be replaced by — sinh’ x, and vice versa. Note that this 
replacement is necessary even if the sin? x is hidden, e.g. tan? x = sin? x/ cos? x 


and so must be replaced by (— sinh? x/ cosh? x) = — tanh’ x. 


> Find the hyperbolic identity analogous to cos? x + sin’ x = 1. 


Using the rules stated above cos” x is replaced by cosh? x, and sin? x by — sinh’ x, and so 
the identity becomes 


cosh’ x — sinh’ x = 1. 


This can be verified by direct substitution, using the definitions of coshx and sinh x; see 
(3.38) and (3.39). < 


Some other identities that can be proved in a similar way are 


sech*x = 1 — tanh’ x, (3.48) 
cosech*x = coth? x — 1, (3.49) 
sinh 2x = 2 sinh x cosh x, (3.50) 
cosh 2x = cosh’ x + sinh? x. (3.51) 
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3.7.4 Solving hyperbolic equations 


When we are presented with a hyperbolic equation to solve, we may proceed 
by analogy with the solution of trigonometric equations. However, it is almost 
always easier to express the equation directly in terms of exponentials. 


> Solve the hyperbolic equation coshx — 5 sinhx — 5 = 0. 


Substituting the definitions of the hyperbolic functions we obtain 


5(e* +e%*)— S(e* —e%*)—5=0. 


Rearranging, and then multiplying through by —e*, gives in turn 
—2e* + 3e* —5=0 
and 
2e* + 5e°—3=0. 
Now we can factorise and solve: 
(2e* — 1)(e* + 3) =0. 


Thus e* = 1/2 or e* = —3. Hence x = —In2 or x = In(—3). The interpretation of the 
logarithm of a negative number has been discussed in section 3.5. < 


3.7.5 Inverses of hyperbolic functions 


Just like trigonometric functions, hyperbolic functions have inverses. If y = 
coshx then x = cosh! y, which serves as a definition of the inverse. By using 
the fundamental definitions of hyperbolic functions, we can find closed-form 
expressions for their inverses. This is best illustrated by example. 


> Find a closed-form expression for the inverse hyperbolic function y = sinh x. 


First we write x as a function of y, ie. 
y=sinh'x => x-=sinhy. 
Now, since cosh y = s(e” +e’) and sinhy = 3(e” —e?), 
e” =coshy + sinh y 
= vA + sinh? y + sinh y 
v= J1+x2 +x, 


and hence 


y=In(V14+x2+4+x). < 


In a similar fashion it can be shown that 
cosh! x = In(./x2 —1+4+ x). 
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4, 
| sech7!x 
cosh”! x 
2 le 
4 2 3 4x 
2 ea 
ho > 
| sech7!x ss: : 
—4l 


Figure 3.14 Graphs of cosh”! x and sech7'x. 


> Find a closed-form expression for the inverse hyperbolic function y = tanh"! x. 


First we write x as a function of y, Le. 
y =tanh! x > x =tanhy. 
Now, using the definition of tanh y and rearranging, we find 


ae Se jer sh 4 
~ te? c PES NEES 


Thus, it follows that 


oy ~itx as =f 
1-x’ 


=lnf 
on 1-x’ 


tants = 51 (7+). « 


1-x 


Graphs of the inverse hyperbolic functions are given in figures 3.14—3.16. 


3.7.6 Calculus of hyperbolic functions 


Just as the identities of hyperbolic functions closely follow those of their trigono- 
metric counterparts, so their calculus is similar. The derivatives of the two basic 
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Figure 3.15 Graphs of sinh’ x and cosech™'x. 
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Figure 3.16 Graphs of tanh”! x and coth' x. 


hyperbolic functions are given by 


(cosh x) = sinh x, (3.52) 
< (sinh x) = cosh x. (3.53) 


They may be deduced by considering the definitions (3.38), (3.39) as follows. 
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> Verify the relation (d/dx) coshx = sinhx. 


Using the definition of cosh x, 
coshx = 5(e* +e), 


and differentiating directly, we find 


£ (cosh x) = z(e —e*) 


Clearly the integrals of the fundamental hyperbolic functions are also defined 
by these relations. The derivatives of the remaining hyperbolic functions can be 
derived by product differentiation and are presented below only for complete- 
ness. 


(tanh x) = sech?x, (3.54) 

dx 

d 
ay (sech x) = —sech x tanh x, (3.55) 

x 

1 

x (cosech x) = —cosech x coth x, (3.56) 
< (coth x) = —cosech*x. (3.57) 

x 


The inverse hyperbolic functions also have derivatives, which are given by the 
following: 


d nj 6 1 

Fy (cosh 2) = Pers Gee) 
d x 1 

— (sinh —) = —— 3.59 
dx (sin -) eta ( ) 
d 1X\ a 2 4 

ax (tanh “) = PRO, for x° <a A (3.60) 

< (coth “) = — 2 for x2 > a’. (3.61) 
x x? — 4 


These may be derived from the logarithmic form of the inverse (see subsec- 
tion 3.7.5). 
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> Evaluate (d/dx) sinh! x using the logarithmic form of the inverse. 


From the results of section 3.7.5, 


3.1 


3.2 


3.3 
3.4 


3.5 


3.6 


3.7 


3.8 


Ope d 
7 (sinh oe aear 


x x 


[in (x + Vx +1)| 


3.8 Exercises 


Two complex numbers z and w are given by z = 3+ 4i and w = 2—i. On an 
Argand diagram, plot 

(a) z+w, (b) w—z, (c) wz, (d) z/w, 

(e) z*w+w'z, (f) w’, (g) Inz, (h) (1 +2+w)!. 

By considering the real and imaginary parts of the product ee prove the 
standard formulae for cos(0 + ) and sin(@ + @). 

By writing 2/12 = (2/3) — (1/4) and considering e’7/'!?, evaluate cot(/12). 

Find the locus in the complex z-plane of points that satisfy the following equa- 
tions. 


1—it 
that varies in the range —0 <t<o. 
(b) z=a+bt+ct’, in which t is a real parameter and a, b, and ¢ are complex 
numbers with b/c real. 


1+it F ; é 
(a) z—c=p9 ( ze ) , where c is complex, p is real and ¢ is a real parameter 


Evaluate 
(a) Re(exp 2iz), (b) Im(cosh” z), (c) (—1 + 31)”, 
(d) jexp(i”)|, (e) exp(#), (f) Im(2"*), (g) i, (h) Inf(V¥3 +1)". 
Find the equations in terms of x and y of the sets of points in the Argand 
diagram that satisfy the following: 
(a) Rez? =Imz?; 
(b) (Imz?)/z* = —i; 
(c) arg[z/(z —1)] = 2/2. 
Show that the locus of all points z = x + iy in the complex plane that satisfy 
\z —ia| = Alz+ia|, A>O, 


is a circle of radius |2Aa/(1 — /”)| centred on the point z = ia[(1 + 27)/(1 — 2”)]. 
Sketch the circles for a few typical values of A, including 2 < 1,4>1and/4=1. 

The two sets of points z =a, z = b,z =c, and z = A, z = B,z = C are 
the corners of two similar triangles in the Argand diagram. Express in terms of 
@ Dyce gE 
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3.9 


(a) the equalities of corresponding angles, and 
(b) the constant ratio of corresponding sides, 


in the two triangles. 
By noting that any complex quantity can be expressed as 


z =|z| exp(iargz), 
deduce that 
a(B —C)+b(C — A) +c(A— B) =0. 


For the real constant a find the loci of all points z = x + iy in the complex plane 
that satisfy 


(a) Re{in(2=*) i =<, c>0, 


(b) im {in (2) bg, 0<k<n/2. 


Z+ila 


Identify the two families of curves and verify that in case (b) all curves pass 
through the two points tia. 

The most general type of transformation between one Argand diagram, in the 
z-plane, and another, in the Z-plane, that gives one and only one value of Z for 
each value of z (and conversely) is known as the general bilinear transformation 
and takes the form 


(a) Confirm that the transformation from the Z-plane to the z-plane is also a 
general bilinear transformation. 
(b) Recalling that the equation of a circle can be written in the form 


2 2Z{ 


=% AeA 


Z—Z2 


show that the general bilinear transformation transforms circles into circles 
(or straight lines). What is the condition that z;, z. and 4 must satisfy if the 
transformed circle is to be a straight line? 


Sketch the parts of the Argand diagram in which 

(a) Rez? <0, |z!/?| <2; 

(b) O<argz* < 2/2; 

(c) |expz*| > 0 as |z| > o. 

What is the area of the region in which all three sets of conditions are satisfied? 
Denote the nth roots of unity by 1, On, @2,..., of"! 


n 


(a) Prove that 


n—1 n—1 


@ SJo=0, di) JJo=Cn™. 
r=0 


r=0 


(b) Express x? + y? +z? — yz—zx —xy as the product of two factors, each linear 
in x, y and z, with coefficients dependent on the third roots of unity (and 
those of the x terms arbitrarily taken as real). 
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Prove that x?”"*! — q?"*!, where m is an integer > 1, can be written as 


xemtl _ g2mtl — (x — a) Il ls — 2ax cos (<= :) + a : 


r=1 


The complex position vectors of two parallel interacting equal fluid vortices 
moving with their axes of rotation always perpendicular to the z-plane are 2, 
and z>. The equations governing their motions are 

dz; i dz; i 

dt Zy— 22 dt Za —Z1 


Deduce that (a) z; + 2, (b) |z; — z2| and (c) |z;|? + |z2|* are all constant in time, 
and hence describe the motion geometrically. 
Solve the equation 

z’ — 42° + 62° — 624 + 623 — 1227 + 82 +4=0, 
(a) by examining the effect of setting z> equal to 2, and then 
(b) by factorising and using the binomial expansion of (z + a)*. 
Plot the seven roots of the equation on an Argand plot, exemplifying that complex 
roots of a polynomial equation always occur in conjugate pairs if the polynomial 
has real coefficients. 
The polynomial f(z) is defined by 

f(z) = 2 — 624 + 1523 — 342? + 362 — 48. 


(a) Show that the equation f(z) = 0 has roots of the form z = Ai, where / is 
real, and hence factorize f(z). 

(b) Show further that the cubic factor of f(z) can be written in the form 
(z + a)} +b, where a and b are real, and hence solve the equation f(z) = 0 
completely. 


The binomial expansion of (1 + x)", discussed in chapter 1, can be written for a 
positive integer n as 
n 


+x)" =o"Gx, 


r=0 
where "C, = n!/[r\(n—r)!]. 
(a) Use de Moivre’s theorem to show that the sum 
Si(n) ="Co — "Co + "Cg — +++ + (-1)" "Com, n—1<2m<n, 
has the value 2”? cos(nz/4). 
(b) Derive a similar result for the sum 
So(n) = "Cy —"C3 + "C5 — +++ + (-1)" "Comat, n—1<2m4+1<n, 
and verify it for the cases n = 6, 7 and 8. 
By considering (1 + exp iO)", prove that 
3 "C, cosr0 = 2" cos"(0/2) cos(n0/2), 
r=0 
> "C, sinr@ = 2” cos"(@/2) sin(n0/2), 


r=0 
where "C, = n!/[rl(n—r)!]. 
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3.20 


3.21 


3.22 


3.23 


3.24 


3.25 


3.26 


Use de Moivre’s theorem with n = 4 to prove that 


cos 40 = 8cost 0 — 8cos? 0 + 1, 


fe 24/3 1/2 
cos F = m 


Express sin‘ @ entirely in terms of the trigonometric functions of multiple angles 


and deduce that its average value over a complete cycle is 3. 


8 
Use de Moivre’s theorem to prove that 

0° — 1003 + St 

5t4 — 1022 + 1’ 

where t = tan 0. Deduce the values of tan(nz/10) for n = 1, 2, 3, 4. 
Prove the following results involving hyperbolic functions. 


(a) That 


and deduce that 


tan 50 = 


cosh x — cosh y = 2sinh (5) sinh (=) F 


(b) That, if y = sinh™! x, 
ay dy 
2 
(x° + Naa 5 +X °F =0. 
Determine the conditions under which the equation 
acoshx + bsinhx =c, c>0, 


has zero, one, or two real solutions for x. What is the solution if a? = c? + b?? 
Use the definitions and properties of hyperbolic functions to do the following: 


(a) Solve coshx = sinh x + 2sech x. 

(b) Show that the real solution x of tanhx = cosech x can be written in the 
form x = In(u + ,/u). Find an explicit value for wu. 

(c) Evaluate tanhx when x is the real solution of cosh2x = 2 coshx. 


Express sinh* x in terms of hyperbolic cosines of multiples of x, and hence find 
the real solutions of 


2cosh4x — 8cosh2x +5 =0. 


In the theory of special relativity, the relationship between the position and time 
coordinates of an event, as measured in two frames of reference that have parallel 
x-axes, can be expressed in terms of hyperbolic functions. If the coordinates are 
x and ¢ in one frame and x’ and f¢’ in the other, then the relationship take the 
form 

x’ = xcosh d — ct sinh ¢, 

ct’ = —x sinh d + ct cosh @. 
Express x and ct in terms of x’, ct’ and ¢ and show that 


x? — (ct? = (xP — (ety. 
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3.27 


3.28 


3.1 


3.3 


3.5 


3.7 


3.9 


3.11 
3.13 


3.15 
3.17 
3.19 
3.21 


3.23 


3.25 


A closed barrel has as its curved surface the surface obtained by rotating about 
the x-axis the part of the curve 


y = a2 —cosh(x/a)] 


lying in the range —b < x < b, where b < acosh”' 2. Show that the total surface 
area, A, of the barrel is given by 
A = na[9a — 8aexp(—b/a) + aexp(—2b/a) — 2b]. 


The principal value of the logarithmic function of a complex variable is defined 
to have its argument in the range —z < arg z < a. By writing z = tanw in terms 


of exponentials show that 
1 1+iz 
antz = — 
tan z zn (748), 


tan”! aS — *) 
7 : 


Use this result to evaluate 


3.9 Hints and answers 


(a) 5+ 31; (b) —1 — Si; (c) 10 + Si; (d) 2/5 + 1171/5; (e) 4; (f) 3 — 43; 

(g) In5 + i[tan-!(4/3) + 2nn]; (h) +(2.521 + 0.595i). 

Use sinz/4 = cos1/4 = 1/./2, sinz/3 = 1/2 and cosz/3 = (3/2. 

cotn/12 =2+ ,/3. 

(a) exp(—2y) cos 2x; (b) (sin 2y sinh 2x)/2; (c) ./2exp(zi/3) or ,/2 exp(47i/3); 

(d) exp(1/./2) or exp(—1/./2); (e) 0.540 — 0.841i; (f) 8 sin(In 2) = 5.11; 

(g) exp(—z/2 — 2zn); (h) In8 + i(6n + 1/2)z. 

Starting from |x + iy — ia| = A|x + iy + ia|, show that the coefficients of x and y 

are equal, and write the equation in the form x? +(y— a)? =r’. 

(a) Circles enclosing z = —ia, with A = expec > 1. 

(b) The condition is that arg[(z—ia)/(z+ia)] =k. This can be rearranged to give 
a(z + 2") = (a? — |z|’) tank, which becomes in x, y coordinates the equation 
of a circle with centre (—acotk,0) and radius acosec k. 

All three conditions are satisfied in 32/2 < 0 < 72/4, |z| < 4; area = 27. 

Denoting exp[2zi/(2m + 1)] by Q, express x?”"*! — q?"*! as a product of factors 

like (x — aQ") and then combine those containing Q" and Q?”*'~’, Use the fact 

that Q?"*! = 1. 

The roots are 2'/ exp(2nni/3) for n = 0,1,2; 14 3'/4; 143'4i. 

Consider (1 + i)". (b) S)(n) = 2"”? sin(nz/4). S2(6) 8, S:(7) = —8, S2(8) = 0. 

Use the binomial expansion of (cos @ + isin 0)‘. 

Show that cos 50 = 16c> — 20c? + 5c, where c = cos 0, and correspondingly for 

sin 50. Use cos~? 9 = 1 + tan? 0. The four required values are 

[(5 — 20)/5]"2, (5 — 20)", [(5 + /20)/5]"2, (5 + /20)'?. 

Reality of the root(s) requires c? + b? > a? and a+b > 0. With these conditions, 

there are two roots if a” > b?, but only one if b? > a’. 

For a = ¢? +b’, x = }In[(a— b)/(a+ d)]. 

Reduce the equation to 16 sinh’ x = 1, yielding x = +0.481. 
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3.27 Show that ds = (cosh x/a) dx; 
curved surface area = ma’[8 sinh(b/a) — sinh(2b/a)] — 2xab. 
flat ends area = 22a?[4 — 4cosh(b/a) + cosh?(b/a)]. 
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Series and limits 


4.1 Series 


Many examples exist in the physical sciences of situations where we are presented 
with a sum of terms to evaluate. For example, we may wish to add the contributions 
from successive slits in a diffraction grating to find the total light intensity at a 
particular point behind the grating. 

A series may have either a finite or infinite number of terms. In either case, the 
sum of the first N terms of a series (often called a partial sum) is written 


Sy =u +up +43 +---+un, 


where the terms of the series un, n = 1,2,3,...,N are numbers, that may in 
general be complex. If the terms are complex then Sy will in general be complex 
also, and we can write Sy = Xy +iYn, where Xy and Yy are the partial sums of 
the real and imaginary parts of each term separately and are therefore real. If a 
series has only N terms then the partial sum Sy is of course the sum of the series. 
Sometimes we may encounter series where each term depends on some variable, 
x, say. In this case the partial sum of the series will depend on the value assumed 
by x. For example, consider the infinite series 


S@x)=1tx4+ 545+ 

This is an example of a power series; these are discussed in more detail in 

section 4.5. It is in fact the Maclaurin expansion of expx (see subsection 4.6.3). 

Therefore S(x) = expx and, of course, varies according to the value of the 
variable x. A series might just as easily depend on a complex variable z. 

A general, random sequence of numbers can be described as a series and a sum 

of the terms found. However, for cases of practical interest, there will usually be 
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some sort of relationship between successive terms. For example, if the nth term 
of a series is given by 


for n= 1,2,3,...,.N then the sum of the first N terms will be 


ee sere rte (4.1) 
N n aaa ad | N° . 
i 2 


It is clear that the sum of a finite number of terms is always finite, provided 
that each term is itself finite. It is often of practical interest, however, to consider 
the sum of a series with an infinite number of finite terms. The sum of an 
infinite number of terms is best defined by first considering the partial sum 
of the first N terms, Sy. If the value of the partial sum Sy tends to a finite 
limit, S, as N tends to infinity, then the series is said to converge and its sum 
is given by the limit S. In other words, the sum of an infinite series is given 
by 


S = lim Sy, 
N-oo 
provided the limit exists. For complex infinite series, if Sy approaches a limit 
S=xX+iY as N > o, this means that X¥,y — X and Yy — Y separately, ie. 
the real and imaginary parts of the series are each convergent series with sums 
X and Y respectively. 

However, not all infinite series have finite sums. As N — oo, the value of the 
partial sum Sy may diverge: it may approach +00 or —oo, or oscillate finitely 
or infinitely. Moreover, for a series where each term depends on some variable, 
its convergence can depend on the value assumed by the variable. Whether an 
infinite series converges, diverges or oscillates has important implications when 
describing physical systems. Methods for determining whether a series converges 
are discussed in section 4.3. 


4.2 Summation of series 


It is often necessary to find the sum of a finite series or a convergent infinite 
series. We now describe arithmetic, geometric and arithmetico-geometric series, 
which are particularly common and for which the sums are easily found. Other 
methods that can sometimes be used to sum more complicated series are discussed 
below. 
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4.2.1 Arithmetic series 


An arithmetic series has the characteristic that the difference between successive 
terms is constant. The sum of a general arithmetic series is written 


N-1 
Sy =a+(a+d)+(a+2d)+---+ [a+(N—1)d] = }\(a + nd). 
n=0 


Rewriting the series in the opposite order and adding this term by term to the 
original expression for Sy, we find 


Sv = > [a+a+t+(N—1)d] = > (hirst term + last term). (4.2) 


If an infinite number of such terms are added the series will increase (or decrease) 
indefinitely; that is to say, it diverges. 


Sum the integers between 1 and 1000 inclusive. 


This is an arithmetic series with a = 1, d = 1 and N = 1000. Therefore, using (4.2) we find 


Sv = on + 1000) = 500500, 


which can be checked directly only with considerable effort. < 


4.2.2 Geometric series 


Equation (4.1) is a particular example of a geometric series, which has the 
characteristic that the ratio of successive terms is a constant (one-half in this 
case). The sum of a geometric series is in general written 


N-1 
Sy =a+tar+ar?+---+arN1 = y ar", 
n=0 


where a is a constant and r is the ratio of successive terms, the common ratio. The 
sum may be evaluated by considering Sy and rSy: 


Sy =a+ar+ar+are+---+arN, 


rSy = ar 4 ar? tar? +art +---+ar™, 


If we now subtract the second equation from the first we obtain 


(1—r)Sy =a—ar%, 


and hence 


_a(l—r¥) 


Sw 1—r 


(4.3) 
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For a series with an infinite number of terms and |r| < 1, we have limy_,..r% = 0, 
and the sum tends to the limit 


(4.4) 


In (4.1), 7 = 5 a= 5 and so S = 1. For |r| > 1, however, the series either diverges 


or oscillates. 


> Consider a ball that drops from a height of 27m and on each bounce retains only a third 
of its kinetic energy; thus after one bounce it will return to a height of 9m, after two 


bounces to 3m, and so on. Find the total distance travelled between the first bounce and 
the Mth bounce. 


The total distance travelled between the first bounce and the Mth bounce is given by the 
sum of M — | terms: 
M-2 


Sy1=2(9+34+14-- y=2y 2 
m=0 


for M > 1, where the factor 2 is included to allow for both the upward and the downward 
journey. Inside the parentheses we clearly have a geometric series with first term 9 and 
common ratio 1/3 and hence the distance is given by (4.3), Le 


9f1—()" 7] 


ree 27 [1 (yr a 


where the number of terms N in (4.3) has been replaced by M —1. < 


Su-1 = 2x 


4.2.3 Arithmetico-geometric series 


An arithmetico-geometric series, as its name suggests, is a combined arithmetic 
and geometric series. It has the general form 


Sy =at(atd)r + (a+ 2dr +---+ [a+ (N —1)d]r or + nd)r”, 
n=0 
and can be summed, in a similar way to a pure geometric series, by multiplying 
by r and subtracting the result from the original series to obtain 


(1—r)Sy =atrd+r°d+---+r%d— [a+ (N — 1d]r® 


Using the expression for the sum of a geometric series (4.3) and rearranging, we 
find 

a—[a+(N—1)d]r% | rd —r%—) 

Pen tee Fae 

For an infinite series with |r| < 1, limy_..rN = 0 as in the previous subsection, 
and the sum tends to the limit 


Sy = 


a rd 


As for a geometric series, if |r| > 1 then the series either diverges or oscillates. 
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> Sum the series 


This is an infinite arithmetico-geometric series with a = 2, d = 3 and r = 1/2. Therefore, 
from (4.5), we obtain S = 10. < 


4.2.4 The difference method 


The difference method is sometimes useful in summing series that are more 
complicated than the examples discussed above. Let us consider the general series 


N 
SO im = uy + uy +++ tu. 


n=1 


If the terms of the series, u,, can be expressed in the form 


un = f(n) — f(n—1) 


for some function f(n) then its (partial) sum is given by 


N 
Sy = om = fN)— FO). 


n=1 


This can be shown as follows. The sum is given by 
Sn =u +u2+---+un 
and since u, = f(n) — f(n — 1), it may be rewritten 
Sw = [f) — FO] + [F2) — FI +--+ FN) — f(N — DI. 
By cancelling terms we see that 


Sy = f(N) — f(0). 


> Evaluate the sum 


Using partial fractions we find 


ber Sel <i 
aon n+1 nj° 


Hence u, = f(n) — f(n — 1) with f(n) = —1/(n+ 1), and so the sum is given by 


1 N 
Sw = f(N) — f(0) wait! ret s 
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The difference method may be easily extended to evaluate sums in which each 
term can be expressed in the form 


un = f(n) —f(n—m), (4.6) 


where m is an integer. By writing out the sum to N terms with each term expressed 
in this form, and cancelling terms in pairs as before, we find 


Sy =o f(N-k +1) 2 k). 


k=1 


> Evaluate the sum 


Using partial fractions we find 


ee (ae ae: 
Oe | GD) In|” 


Hence u, = f(n) — f(n — 2) with f(n) = —1/[2(n + 2)], and so the sum is given by 


Se = FON) +F0N 1) f0)~F- = 3-5 (S54 yRq)-« 

In fact the difference method is quite flexible and may be used to evaluate 
sums even when each term cannot be expressed as in (4.6). The method still relies, 
however, on being able to write u, in terms of a single function such that most 
terms in the sum cancel, leaving only a few terms at the beginning and the end. 
This is best illustrated by an example. 


> Evaluate the sum 


1 
oe n(n + 1)(n + 2)" 


n=1 


Using partial fractions we find 


1 1 1 


n+2) n+l er 

Hence u, = f(n) — 2f(n— 1) + f(n — 2) with f(n) = 1/[2(n + 2)]. If we write out the sum, 
expressing each term uw, in this form, we find that most terms cancel and the sum is given 
by 


Un, = 


1 1 
Sw = f(N) —f(N —1) — f) + f(-1) 3+3(we3-ya)* 
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4.2.5 Series involving natural numbers 


Series consisting of the natural numbers 1, 2, 3, ..., or the square or cube of these 
numbers, occur frequently and deserve a special mention. Let us first consider 
the sum of the first N natural numbers, 


N 
Sy =14+24+34+---+N=Son 


n=1 
This is clearly an arithmetic series with first term a = 1 and common difference 
d = 1. Therefore, from (4.2), Sy = 5N(N + 1). 
Next, we consider the sum of the squares of the first N natural numbers: 


N 
Sy=VP427474...4N => on’, 
n=1 


which may be evaluated using the difference method. The nth term in the series 
is u, =n’, which we need to express in the form f(n)— f(n—1) for some function 
f(n). Consider the function 


f(n) = n(n + 1)(2n + 1) f(n— 1) = (n— 1)n(2n — 1). 
For this function f(n) — f(n— 1) = 6n’, and so we can write 
Un = gL f(n) — f(n—1)]. 
Therefore, by the difference method, 
Sy = gLf(N) — f(0)] = gN(N + DQN +1). 


Finally, we calculate the sum of the cubes of the first N natural numbers, 


N 
Sy= P4243? +---4+N => vr’, 


n=1 
again using the difference method. Consider the function 
f(n) = [n+ DP f(n—1) = [(n— Dn’, 


for which f(n) — f(n — 1) = 4n>. Therefore we can write the general nth term of 
the series as 


Un = 4Lf(n) — f(n—1)], 
and using the difference method we find 
Sw = gLf(N) — fO)] = gN*(N + 1). 
Note that this is the square of the sum of the natural numbers, Le. 


ne 


n=1 n=1 
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> Sum the series 


N 


So(nt (nt 3). 


n=1 


The nth term in this series is 
uy = (n+ 1)(n+ 3) = 7? +4n4 3, 
and therefore we can write 


N N 
Son t Dnt 3) = So? + 4n + 3) 


n=1 n=1 


N N N 
So +45 ont 503 
n=1 


n=1 n=1 


= tN(N + 1I)(QQN +1) +4 SN(N +1) 4+3N 
= 4N(2N? + 15N +31). 


4.2.6 Transformation of series 


A complicated series may sometimes be summed by transforming it into a 
familiar series for which we already know the sum, perhaps a geometric series 
or the Maclaurin expansion of a simple function (see subsection 4.6.3). Various 
techniques are useful, and deciding which one to use in any given case is a matter 
of experience. We now discuss a few of the more common methods. 

The differentiation or integration of a series is often useful in transforming an 
apparently intractable series into a more familiar one. If we wish to differentiate 
or integrate a series that already depends on some variable then we may do so 
in a straightforward manner. 


> Sum the series 


S(x) 


3(0!) + a a 5(2!) sa 


Dividing both sides by x we obtain 


S(x) 3 x4 x 
+ + 
x 3(0!) 41!) 5(2!) 
which is easily differentiated to give 


d [S(x) rea cn a 

dx Oo! it! 2! 3! : 

Recalling the Maclaurin expansion of expx given in subsection 4.6.3, we recognise that 
the RHS is equal to x? exp x. Having done so, we can now integrate both sides to obtain 


fen, 


x 


S(x)/x = ie exp x dx. 
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Integrating the RHS by parts we find 
S(x)/x = x? expx — 2xexpx + 2expx +e, 


where the value of the constant of integration c can be fixed by the requirement that 
S(x)/x = 0 at x =0. Thus we find that c = —2 and that the sum is given by 


S(x) = x? exp x — 2x? expx + 2x exp x — 2x. < 


Often, however, we require the sum of a series that does not depend on a 
variable. In this case, in order that we may differentiate or integrate the series, 
we define a function of some variable x such that the value of this function is 
equal to the sum of the series for some particular value of x (usually at x = 1). 


> Sum the series 


Let us begin by defining the function 
f(x) =142x43x? +407 4---, 
so that the sum S = f(1/2). Integrating this function we obtain 


[i@aaxteeet, 


which we recognise as an infinite geometric series with first term a = x and common ratio 
r =x. Therefore, from (4.4), we find that the sum of this series is x/(1— x). In other words 


x 
[fo a= 


d x 1 
F(x) race (—xp 
The sum of the original series is therefore S = f(1/2)=4. < 


so that f(x) is given by 


Aside from differentiation and integration, an appropriate substitution can 
sometimes transform a series into a more familiar form. In particular, series with 
terms that contain trigonometric functions can often be summed by the use of 
complex exponentials. 


> Sum the series 
cos20  _cos30 


S(0) =1+cos0+ 7 a 31 


Replacing the cosine terms with a complex exponential, we obtain 


S(0) = Re{ 1+ expid + AD 4 XE 4 
2 31 
y2 oy3 
= Re{1 + expié + expt) + (exp) ae i. 
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Again using the Maclaurin expansion of expx given in subsection 4.6.3, we notice that 
S(O) = Re [exp(exp i0)] = Re [exp(cos 0 + isin 0)] 
= Re {[exp(cos 0)][exp(i sin 0)]} = [exp(cos 0)]Re [exp(isin 0)] 
= [exp(cos 0)][cos(sin 0)]. < 


4.3 Convergence of infinite series 


Although the sums of some commonly occurring infinite series may be found, 
the sum of a general infinite series is usually difficult to calculate. Nevertheless, 
it is often useful to know whether the partial sum of such a series converges to 
a limit, even if the limit cannot be found explicitly. As mentioned at the end of 
section 4.1, if we allow N to tend to infinity, the partial sum 


N 
Sn = Se Un 


n=1 
of a series may tend to a definite limit (i.e. the sum S of the series), or increase 
or decrease without limit, or oscillate finitely or infinitely. 
To investigate the convergence of any given series, it is useful to have available 
a number of tests and theorems of general applicability. We discuss them below; 
some we will merely state, since once they have been stated they become almost 
self-evident, but are no less useful for that. 


4.3.1 Absolute and conditional convergence 


Let us first consider some general points concerning the convergence, or otherwise, 
of an infinite series. In general an infinite series }*u, can have complex terms, 
and in the special case of a real series the terms can be positive or negative. From 
any such series, however, we can always construct another series 5> |u,| in which 
each term is simply the modulus of the corresponding term in the original series. 
Then each term in the new series will be a positive real number. 

If the series 5~ |u,| converges then }*u, also converges, and “uy is said to be 
absolutely convergent, i.e. the series formed by the absolute values is convergent. 
For an absolutely convergent series, the terms may be reordered without affecting 
the convergence of the series. However, if > |u,| diverges whilst 5~ u,, converges 
then 5° u, is said to be conditionally convergent. For a conditionally convergent 
series, rearranging the order of the terms can affect the behaviour of the sum 
and, hence, whether the series converges or diverges. In fact, a theorem due 
to Riemann shows that, by a suitable rearrangement, a conditionally convergent 
series may be made to converge to any arbitrary limit, or to diverge, or to oscillate 
finitely or infinitely! Of course, if the original series S~ u, consists only of positive 
real terms and converges then automatically it is absolutely convergent. 
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4.3.2 Convergence of a series containing only real positive terms 


As discussed above, in order to test for the absolute convergence of a series 
So un, we first construct the corresponding series 5~ |u,| that consists only of real 
positive terms. Therefore in this subsection we will restrict our attention to series 
of this type. 

We discuss below some tests that may be used to investigate the convergence of 
such a series. Before doing so, however, we note the following crucial consideration. 
In all the tests for, or discussions of, the convergence of a series, it is not what 
happens in the first ten, or the first thousand, or the first million terms (or any 
other finite number of terms) that matters, but what happens ultimately. 


Preliminary test 


A necessary but not sufficient condition for a series of real positive terms 5° uy, 
to be convergent is that the term u, tends to zero as n tends to infinity, i.e. we 
require 


lim u, = 0. 
n—oo 


If this condition is not satisfied then the series must diverge. Even if it is satisfied, 
however, the series may still diverge, and further testing is required. 


Comparison test 


The comparison test is the most basic test for convergence. Let us consider two 
series }>u, and )>v, and suppose that we know the latter to be convergent (by 
some earlier analysis, for example). Then, if each term u, in the first series is less 
than or equal to the corresponding term v, in the second series, for all n greater 
than some fixed number N that will vary from series to series, then the original 
series )* u, is also convergent. In other words, if 5 v, is convergent and 


Uj<v, forn>N, 


then 5“ u, converges. 
However, if 5>v, diverges and u, > v, for all n greater than some fixed number 
then 5“ u, diverges. 


> Determine whether the following series converges: 


SERIES AND LIMITS 


which is merely the series obtained by setting x = 1 in the Maclaurin expansion of exp x 
(see subsection 4.6.3), i.e. 
1 1 1 
exp(1)=e=1+ Tl +5 +z a toe 
Clearly this second series is convergent, since it consists of only positive terms and has a 
finite sum. Thus, since each term u, in the series (4.7) is less than the corresponding term 
1/n! in (4.8), we conclude from the comparison test that (4.7) is also convergent. < 


D’Alembert’s ratio test 


The ratio test determines whether a series converges by comparing the relative 
magnitude of successive terms. If we consider a series }* u, and set 


penis (“*) (4.9) 


then if p < 1 the series is convergent; if p > 1 the series is divergent; if p = 1 
then the behaviour of the series is undetermined by this test. 

To prove this we observe that if the limit (4.9) is less than unity, Le. p < 1 then 
we can find a value r in the range p <r < 1 and a value N such that 


Un+1 
— <I 
Un 


for all n > N. Now the terms u, of the series that follow uy are 
uUN+1, UN+2; UN+35 oo) 
and each of these is less than the corresponding term of 
ruy, uy, PUN,  .--- (4.10) 


However, the terms of (4.10) are those of a geometric series with a common 
ratio r that is less than unity. This geometric series consequently converges and 
therefore, by the comparison test discussed above, so must the original series 
youn. An analogous argument may be used to prove the divergent case when 


> Determine whether the following series converges: 


As mentioned in the previous example, this series may be obtained by setting x = 1 in the 

Maclaurin expansion of expx, and hence we know already that it converges and has the 

sum exp(1) = e. Nevertheless, we may use the ratio test to confirm that it converges. 
Using (4.9), we have 


: n! : 1 
rtm pti - saath) -9 a 


and since p < 1, the series converges, as expected. < 
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Ratio comparison test 


As its name suggests, the ratio comparison test is a combination of the ratio and 
comparison tests. Let us consider the two series }*u, and )*v, and assume that 
we know the latter to be convergent. It may be shown that if 


Un+1 < Unt 
Un Vn 


for all n greater than some fixed value N then S~u, is also convergent. 
Similarly, if 


Unt Un+1 
n+ > n+ 
Un Vy 


for all sufficiently large n, and )~v, diverges then )*u, also diverges. 


> Determine whether the following series converges: 


In this case the ratio of successive terms, as n tends to infinity, is given by 


oe: it) 4 ie 
sos fea lim (5) : 


which is less than the ratio seen in (4.11). Hence, by the ratio comparison test, the series 
converges. (It is clear that this series could also be found to be convergent using the ratio 
test.) < 


Quotient test 


The quotient test may also be considered as a combination of the ratio and 
comparison tests. Let us again consider the two series }* u, and )~v,, and define 
p as the limit 


p= lim (*) (4.12) 


noo \ Dy 


Then, it can be shown that: 


(i) if p #0 but is finite then 5*u, and 5°», either both converge or both 
diverge; 


(ii) if p =0 and *v, converges then S*u, converges; 


(iii) if p = co and )*»v, diverges then )*u, diverges. 
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> Given that the series >, 1/n diverges, determine whether the following series converges: 


5 dP ap 3 
En 15) 


n=1 


If we set u, = (4n? — n— 3)/(n3 + 2n) and v, = 1/n then the limit (4.12) becomes 
(4n? — n— 3)/(n> + 2n) 4n3 — nr? —3n) _ 
1/n n+2n = 


Since p is finite but non-zero and 5° v, diverges, from (i) above }> u, must also diverge. < 


= lim 


no 


p= lim | 


no 


Integral test 


The integral test is an extremely powerful means of investigating the convergence 
of a series }>u,. Suppose that there exists a function f(x) which monotonically 
decreases for x greater than some fixed value xo and for which f(n) = up, i.e. the 
value of the function at integer values of x is equal to the corresponding term 
in the series under investigation. Then it can be shown that, if the limit of the 
integral 


N 
lim ‘(x)d 
N>o LAX) 
exists, the series }~u, is convergent. Otherwise the series diverges. Note that the 


integral defined here has no lower limit; the test is sometimes stated with a lower 
limit, equal to unity, for the integral, but this can lead to unnecessary difficulties. 


> Determine whether the following series converges: 


ss : qaeges 
(n— 3/2 9° 25 


n=1 


Let us consider the function f(x) = (x —3/2)~?. Clearly f(n) = u, and f(x) monotonically 
decreases for x > 3/2. Applying the integral test, we consider 


im fo ee ee = =0 
Noof (x—3/2P Noe \N — 3/2 ; 


Since the limit exists the series converges. Note, however, that if we had included a lower 
limit, equal to unity, in the integral then we would have run into problems, since the 
integrand diverges at x = 3/2. < 


The integral test is also useful for examining the convergence of the Riemann 
zeta series. This is a special series that occurs regularly and is of the form 


It converges for p > 1 and diverges if p < 1. These convergence criteria may be 
derived as follows. 
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Using the integral test, we consider 


N 1—p 
lim a= lim (= i 


N00 xP Noo \1l—p 


and it is obvious that the limit tends to zero for p > 1 and to 00 for p< 1. 


Cauchy’s root test 


Cauchy’s root test may be useful in testing for convergence, especially if the nth 
terms of the series contains an nth power. If we define the limit 


p = lim (ina)? 
noo 


then it may be proved that the series }*u, converges if p < 1. If p > 1 then the 
series diverges. Its behaviour is undetermined if p = 1. 


> Determine whether the following series converges: 


Using Cauchy’s root test, we find 


. 1 
pam tin (3) me 


Grouping terms 


and hence the series converges. < 


We now consider the Riemann zeta series, mentioned above, with an alternative 
proof of its convergence that uses the method of grouping terms. In general there 
are better ways of determining convergence, but the grouping method may be 
used if it is not immediately obvious how to approach a problem by a better 
method. 

First consider the case where p > 1, and group the terms in the series as 


follows: 
se al one 1 1 
iT ae a t 3p + prota pena 


Now we can see that each bracket of this series is less than each term of the 
geometric series 
1 2: 4 


This geometric series has common ratio r = (rs since p > 1, it follows that 


r <1 and that the geometric series converges. Then the comparison test shows 
that the Riemann zeta series also converges for p > 1. 


129 


SERIES AND LIMITS 


The divergence of the Riemann zeta series for p < 1 can be seen by first 
considering the case p = 1. The series is 


faa CL as 2ft 
ae Se es ats 
Sw tz tagtgt > 


which does not converge, as may be seen by bracketing the terms of the series in 
groups in the following way: 


N 
v= omat+(5) +($+3) +($+3+7+5) foes, 
The sum of the terms in each bracket is > ; and, since as many such groupings 
can be made as we wish, it is clear that Sy increases indefinitely as N is increased. 

Now returning to the case of the Riemann zeta series for p < 1, we note that 
each term in the series is greater than the corresponding one in the series for 
which p = 1. In other words 1/n? > 1/n for n > 1, p < 1. The comparison test 
then shows us that the Riemann zeta series will diverge for all p < 1. 


4.3.3 Alternating series test 


The tests discussed in the last subsection have been concerned with determining 
whether the series of real positive terms S~ |u,| converges, and so whether 5° u, 
is absolutely convergent. Nevertheless, it is sometimes useful to consider whether 
a series is merely convergent rather than absolutely convergent. This is especially 
true for series containing an infinite number of both positive and negative terms. 
In particular, we will consider the convergence of series in which the positive and 
negative terms alternate, i.e. an alternating series. 
An alternating series can be written as 
foe) 
S( 1) uy, = uy — uy $3 — ug tus—--, 


n=1 


with all u, > 0. Such a series can be shown to converge provided (1) u, — 0 as 
n— oo and (il) u, < up_; for alln > N for some finite N. If these conditions are 
not met then the series oscillates. 

To prove this, suppose for definiteness that N is odd and consider the series 
starting at uy. The sum of its first 2m terms is 


Som = (un — Un41) + (UN42 — UN43) +++ + (UN+2m—2 — UN-+42m-1)- 


By condition (ii) above, all the parentheses are positive, and so Sy», increases as 
m increases. We can also write, however, 


Som = Un — (UN41 — UN42) — +++ — (UN42m—3 — UN+2m—2) — UN-42m—1s 


and since each parenthesis is positive, we must have S>,, < uy. Thus, since S,, 
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is always less than uy for all m and u, — 0 as n > o, the alternating series 
converges. It is clear that an analogous proof can be constructed in the case 
where N is even. 


> Determine whether the following series converges: 


oe) 


1 
n+1 
(-1y— 


This alternating series clearly satisfies conditions (i) and (ii) above and hence converges. 
However, as shown above by the method of grouping terms, the corresponding series with 
all positive terms is divergent. < 


4.4 Operations with series 


Simple operations with series are fairly intuitive, and we discuss them here only 
for completeness. The following points apply to both finite and infinite series 
unless otherwise stated. 


(i) If Sou, = S then $> ku, =kS where k is any constant. 

(ii) If Su, = S and S*v, = T then S\(u, + 0p) =S+ T. 

(iii) If Sou, = S then a+ )>u, = a+S. A simple extension of this trivial result 
shows that the removal or insertion of a finite number of terms anywhere 
in a series does not affect its convergence. 

(iv) If the infinite series )>u, and S>v, are both absolutely convergent then 
the series }* wy», where 


Wr = U1Un + U2Vp—1 +++ + Unr1, 


is also absolutely convergent. The series }~ w, is called the Cauchy product 
of the two original series. Furthermore, if }>u, converges to the sum S 
and 5+ v, converges to the sum T then }*>w, converges to the sum ST. 
It is not true in general that term-by-term differentiation or integration of 
a series will result in a new series with the same convergence properties. 


—~S 


(v 


4.5 Power series 
A power series has the form 
P(x) =agp + ax+@x° +axert---, 


where do, 41, a, a3 etc. are constants. Such series regularly occur in physics and 
engineering and are useful because, for |x| < 1, the later terms in the series may 
become very small and be discarded. For example the series 


P(x) =14+x4+x?4x34---, 
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although in principle infinitely long, in practice may be simplified if x happens to 
have a value small compared with unity. To see this note that P(x) for x = 0.1 
has the following values: 1, if just one term is taken into account; 1.1, for two 
terms; 1.11, for three terms; 1.111, for four terms, etc. If the quantity that it 
represents can only be measured with an accuracy of two decimal places, then all 
but the first three terms may be ignored, i.e. when x = 0.1 or less 


P(x) =14+x+x7+0(°) &14+x42x’. 


This sort of approximation is often used to simplify equations into manageable 
forms. It may seem imprecise at first but is perfectly acceptable insofar as it 
matches the experimental accuracy that can be achieved. 

The symbols O and = used above need some further explanation. They are 
used to compare the behaviour of two functions when a variable upon which 
both functions depend tends to a particular limit, usually zero or infinity (and 
obvious from the context). For two functions f(x) and g(x), with g positive, the 
formal definitions of the above symbols are as follows: 


(i) If there exists a constant k such that |f| < kg as the limit is approached 
then f = O(g). 

(ii) If as the limit of x is approached f/g tends to a limit /, where | # 0, then 
f =~ lg. The statement f ~ g means that the ratio of the two sides tends 
to unity. 


4.5.1 Convergence of power series 


The convergence or otherwise of power series is a crucial consideration in practical 
terms. For example, if we are to use a power series as an approximation, it is 
clearly important that it tends to the precise answer as more and more terms of 
the approximation are taken. Consider the general power series 


P(x) = tayx +apx*+---. 


Using d’Alembert’s ratio test (see subsection 4.3.2), we see that P(x) converges 
absolutely if 


An+1 
an 


a. , 
att | = |x| lim eo: 


noo 


p= lim 


no 


an 


Thus the convergence of P(x) depends upon the value of x, i.e. there is, in general, 
a range of values of x for which P(x) converges, an interval of convergence. Note 
that at the limits of this range p = 1, and so the series may converge or diverge. 
The convergence of the series at the end-points may be determined by substituting 
these values of x into the power series P(x) and testing the resulting series using 
any applicable method (discussed in section 4.3). 
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> Determine the range of values of x for which the following power series converges: 


P(x) =1+2x+4x?+ 8x3 +---. 


By using the interval-of-convergence method discussed above, 


antl 
Qn 7 


p= lim = |2x|, 


no 


and hence the power series will converge for |x| < 1/2. Examining the end-points of the 
interval separately, we find 


P(1/2)=14+1414-., 
P(-1/2)=1—-1+1-—--. 


Obviously P(1/2) diverges, while P(—1/2) oscillates. Therefore P(x) is not convergent at 
either end-point of the region but is convergent for -1<x<1.< 


The convergence of power series may be extended to the case where the 
parameter z is complex. For the power series 


P(z) =a) tayz 4apz7 +-:-, 
we find that P(z) converges if 


Anti 
—Z 
an 


n+ 
an 


= |z| lim Pe 


no 


p= lim 


noo 


We therefore have a range in |z| for which P(z) converges, i.e. P(z) converges 
for values of z lying within a circle in the Argand diagram (in this case centred 
on the origin of the Argand diagram). The radius of the circle is called the 
radius of convergence: if z lies inside the circle, the series will converge whereas 
if z lies outside the circle, the series will diverge; if, though, z lies on the circle 
then the convergence must be tested using another method. Clearly the radius of 
convergence R is given by 1/R = limy 4 |dni1/ay|. 


> Determine the range of values of z for which the following complex power series converges: 


th Re GR 
1 Neon ean erp tS 
@) Fisaer ease a 


We find that p = |z/2|, which shows that P(z) converges for |z| < 2. Therefore the circle 
of convergence in the Argand diagram is centred on the origin and has a radius R = 2. 
On this circle we must test the convergence by substituting the value of z into P(z) and 
considering the resulting series. On the circle of convergence we can write z = 2expi0. 
Substituting this into P(z), we obtain 


2expi0 4 4exp 2i0 
2 4 
= 1-—expid + [expid]}?—---, 


P(z)=1— 


which is a complex infinite geometric series with first term a = 1 and common ratio 
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r =—expi0@. Therefore, on the the circle of convergence we have 
1 
P(z) = ———.. 
(2) 1+expid 
Unless @ = z this is a finite complex number, and so P(z) converges at all points on the 
circle |z| = 2 except at 0 = zm (ie. z = —2), where it diverges. Note that P(z) is just the 
binomial expansion of (1 + z/2)~', for which it is obvious that z = —2 is a singular point. 


In general, for power series expansions of complex functions about a given point in the 
complex plane, the circle of convergence extends as far as the nearest singular point. This 
is discussed further in chapter 24. < 


Note that the centre of the circle of convergence does not necessarily lie at the 
origin. For example, applying the ratio test to the complex power series 
—1 (z-1yP (z—1) 


z 
P fed cics 
(2) : 2 4 8 ; 


we find that for it to converge we require |(z — 1)/2| < 1. Thus the series converges 
for z lying within a circle of radius 2 centred on the point (1,0) in the Argand 
diagram. 


4.5.2 Operations with power series 


The following rules are useful when manipulating power series; they apply to 
power series in a real or complex variable. 

(i) If two power series P(x) and Q(x) have regions of convergence that overlap 
to some extent then the series produced by taking the sum, the difference or the 
product of P(x) and Q(x) converges in the common region. 

(ii) If two power series P(x) and Q(x) converge for all values of x then one 
series may be substituted into the other to give a third series, which also converges 
for all values of x. For example, consider the power series expansions of sin x and 
e* given below in subsection 4.6.3, 


PS x3 x5 ie 
SS ea are ae 
a 2 x3 x4 

e= DT ag ag ae 


both of which converge for all values of x. Substituting the series for sinx into 
that for e* we obtain 
2 4 5 
sine <=. x 3x 8x 
ee ac amy (ee aan 
which also converges for all values of x. 

If, however, either of the power series P(x) and Q(x) has only a limited region 
of convergence, or if they both do so, then further care must be taken when 
substituting one series into the other. For example, suppose Q(x) converges for 
all x, but P(x) only converges for x within a finite range. We may substitute 
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Q(x) into P(x) to obtain P(Q(x)), but we must be careful since the value of Q(x) 
may lie outside the region of convergence for P(x), with the consequence that the 
resulting series P(Q(x)) does not converge. 

(iii) If a power series P(x) converges for a particular range of x then the series 
obtained by differentiating every term and the series obtained by integrating every 
term also converge in this range. 

This is easily seen for the power series 


P(x) = ap ayx tax? +--+, 


which converges if |x| < limps |dn/dn41| = k. The series obtained by differenti- 
ating P(x) with respect to x is given by 


iP 
ea = ay + 2ayx + 3a3x? +--+ 
dx 
and converges if 
: nan 
< lim | ———__] = k. 
Ix as (n+ 1)dy44 
Similarly the series obtained by integrating P(x) term by term, 
2 3 
ayx a2x 
P(x) dx = f peace 
ij (x) dx = aox 5) 3 


converges if 
(n+ 2)an | _ 
(n+ Van+1 


So, series resulting from differentiation or integration have the same interval of 


convergence as the original series. However, even if the original series converges 
at either end-point of the interval, it is not necessarily the case that the new series 
will do so. The new series must be tested separately at the end-points in order 
to determine whether it converges there. Note that although power series may be 
integrated or differentiated without altering their interval of convergence, this is 
not true for series in general. 

It is also worth noting that differentiating or integrating a power series term 
by term within its interval of convergence is equivalent to differentiating or 
integrating the function it represents. For example, consider the power series 
expansion of sin x, 


sngax—- SE, (4.14) 
which converges for all values of x. If we differentiate term by term, the series 
becomes 

x? xt x6 


which is the series expansion of cos x, as we expect. 
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4.6 Taylor series 


Taylor’s theorem provides a way of expressing a function as a power series in x, 
known as a Taylor series, but it can be applied only to those functions that are 
continuous and differentiable within the x-range of interest. 


4.6.1 Taylor’s theorem 


Suppose that we have a function f(x) that we wish to express as a power series 
in x —a about the point x = a. We shall assume that, in a given x-range, f(x) 
is a continuous, single-valued function of x having continuous derivatives with 
respect to x, denoted by f’(x), f’(x) and so on, up to and including f(x). We 
shall also assume that f(x) exists in this range. 

From the equation following (2.31) we may write 


ath 
/ f'@)dx = fath)— fla), 


where a, a+h are neighbouring values of x. Rearranging this equation, we may 
express the value of the function at x = a+h in terms of its value at a by 


a+h 
f(a +h) = f(a) +f f'(x) dx. (4.15) 


A 


A first approximation for f(a +h) may be obtained by substituting f’(a) for 
f'(x) in (4.15), to obtain 


fla +h) & fla) + hf'(@). 


This approximation is shown graphically in figure 4.1. We may write this first 
approximation in terms of x and a as 


f(x) © f(a) + (x— f(a), 
and, in a similar way, 


fo) x f@+e—af"@, 
f") © f(a) +x — a) f(a), 


and so on. Substituting for f’(x) in (4.15), we obtain the second approximation: 


a+h 
flat h) ~ f(a) + / [f'(a) + «—a)f"(a)] ax 


h 
~ fa) +hf(a) + +f" (a). 


We may repeat this procedure as often as we like (so long as the derivatives 
of f(x) exist) to obtain higher-order approximations to f(a +h); we find the 
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FO) 


Figure 4.1 The first-order Taylor series approximation to a function f(x). 
The slope of the function at P, ie. tan 0, equals f’(a). Thus the value of the 
function at Q, f(a+h), is approximated by the ordinate of R, f(a) + hf’(a). 


(n — 1)th-order approximation’ to be 


s “/ I “It ne -(n—1) 
flath) = f(a +hf'(a4 a (a)+-+-4 oil (a). (4.16) 


As might have been anticipated, the error associated with approximating f(a+h) 
by this (n — 1)th-order power series is of the order of the next term in the series. 
This error or remainder can be shown to be given by 


Re(h) = “HE, 


for some € that lies in the range [a,a +h]. Taylor’s theorem then states that we 
may write the equality 


(n—1) 


2, 
flat h) = fla) + hf'(a) + Ea) +--+ A f(a) + Ryl, 
2 (a—1)! A, 


The theorem may also be written in a form suitable for finding f(x) given 
the value of the function and its relevant derivatives at x = a, by substituting 


S The order of the approximation is simply the highest power of h in the series. Note, though, that 
the (n — 1)th-order approximation contains n terms. 
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x =a+h in the above expression. It then reads 


oie e\2 _ g)yn-l 
Flos) = fla) + (= af (a+ SS" pra) + + FB pHa) + Ryo, 
, (4.18) 
where the remainder now takes the form 
Re) = FF pop, 


and € lies in the range [a,x]. Each of the formulae (4.17), (4.18) gives us the 
Taylor expansion of the function about the point x = a. A special case occurs 
when a = 0. Such Taylor expansions, about x = 0, are called Maclaurin series. 

Taylor’s theorem is also valid without significant modification for functions 
of a complex variable (see chapter 24). The extension of Taylor’s theorem to 
functions of more than one variable is given in chapter 5. 

For a function to be expressible as an infinite power series we require it to be 
infinitely differentiable and the remainder term R,, to tend to zero as n tends to 
infinity, 1.e. lim,_,.. R, = 0. In this case the infinite power series will represent the 
function within the interval of convergence of the series. 


> Expand f(x) =sinx as a Maclaurin series, i.e. about x = 0. 


We must first verify that sinx may indeed be represented by an infinite power series. It is 
easily shown that the nth derivative of f(x) is given by 


f(x) = sin (x + >) : 

Therefore the remainder after expanding f(x) as an (n — 1)th-order polynomial about 
x = 0 is given by 

xe nt 
R, =—sI ( =) > 
i") n! amet 2 
where € lies in the range [0, x]. Since the modulus of the sine term is always less than or 
equal to unity, we can write |R,(x)| < |x”"|/n!. For any particular value of x, say x = c, 
R,(c) — 0 as n > o. Hence lim,_,,. R,(x) = 0, and so sinx can be represented by an 
infinite Maclaurin series. 

Evaluating the function and its derivatives at x = 0 we obtain 


f(0) = sind = 0, 
f'(0) = sin(x/2) = 1, 
f"(0) = sina =0, 
f (0) = sin(3z/2) = —-1, 
and so on. Therefore, the Maclaurin series expansion of sin x is given by 
; IP SP 
simxX =X— s+ tee 


Note that, as expected, since sin x is an odd function, its power series expansion contains 
only odd powers of x. < 
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We may follow a similar procedure to obtain a Taylor series about an arbitrary 
point x =a. 


> Expand f(x) =cosx as a Taylor series about x = 1/3. 


As in the above example, it is easily shown that the nth derivative of f(x) is given by 
(n) = nt 
f'" (x) = cos (x + 5) ) ‘ 


Therefore the remainder after expanding f(x) as an (n — 1)th-order polynomial about 
x = 7/3 is given by 


Ry(x) = SAM cos (2 4 ME), 


where € lies in the range [z/3,x]. The modulus of the cosine term is always less than or 
equal to unity, and so |R,(x)| < |(x—72/3)"|/n!. As in the previous example, lim,_,.. Ri(x) = 
0 for any particular value of x, and so cos x can be represented by an infinite Taylor series 
about x = 7/3. 

Evaluating the function and its derivatives at x = 2/3 we obtain 


(2/3) = cos(x/3) = 1/2, 


f'(n/3) = cos(5n/6) = —J3/2, 
f" (2/3) = cos(4n/3) = —1/2, 


and so on. Thus the Taylor series expansion of cos x about x = 7/3 is given by 


.< 


4.6.2 Approximation errors in Taylor series 


In the previous subsection we saw how to represent a function f(x) by an infinite 
power series, which is exactly equal to f(x) for all x within the interval of 
convergence of the series. However, in physical problems we usually do not want 
to have to sum an infinite number of terms, but prefer to use only a finite number 
of terms in the Taylor series to approximate the function in some given range 
of x. In this case it is desirable to know what is the maximum possible error 
associated with the approximation. 

As given in (4.18), a function f(x) can be represented by a finite (n — 1)th-order 
power series together with a remainder term such that 
(x — a)’ (x — a)! 


Te ae ——__ f""V(a) + Rul), 


fix) = f@ +(x—a)f'(at Gayl 


where 
(x—a)" , 
R,(x) = Saat 
and é lies in the range [a, x]. R,(x) is the remainder term, and represents the error 


in approximating f(x) by the above (n — 1)th-order power series. Since the exact 
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value of € that satisfies the expression for R,(x) is not known, an upper limit on 
the error may be found by differentiating R,(x) with respect to € and equating 
the derivative to zero in the usual way for finding maxima. 


> Expand f(x) = cosx as a Taylor series about x = 0 and find the error associated with 
using the approximation to evaluate cos(0.5) if only the first two non-vanishing terms are 


taken. (Note that the Taylor expansions of trigonometric functions are only valid for angles 
measured in radians. ) 


Evaluating the function and its derivatives at x = 0, we find 


f(0) =cos0 = 1, 
f'(0) =—sin0 =0, 
f"(0) =—cos0 = -1, 
f’’ (0) = sin0 = 0. 
So, for small |x|, we find from (4.18) 
42 
cosx x 1— 7 


Note that since cosx is an even function, its power series expansion contains only even 
powers of x. Therefore, in order to estimate the error in this approximation, we must 
consider the term in x*, which is the next in the series. The required derivative is f(x) 
and this is (by chance) equal to cos x. Thus, adding in the remainder term R4(x), we find 


2 4 
cosx = 1-5 + Feosé, 


where € lies in the range [0,x]. Thus, the maximum possible error is x*/4!, since cos é 
cannot exceed unity. If x = 0.5, taking just the first two terms yields cos(0.5) ~ 0.875 with 
a predicted error of less than 0.002 60. In fact cos(0.5) = 0.87758 to 5 decimal places. Thus, 
to this accuracy, the true error is 0.00258, an error of about 0.3%. < 


4.6.3 Standard Maclaurin series 


It is often useful to have a readily available table of Maclaurin series for standard 
elementary functions, and therefore these are listed below. 


: x3 x? x! 
sinx =x aptsrc qt for —o <x< om, 
lay 2 yp SANG ; 
CST a ep Or —-0<x<0, 
3 5 7 
afr ob - x x x es 
tan x=x 3 5 7 for -l<x<1l, 
x? x3 x4 
e=14+x4 7 - ata t-++ for —0 <x <0, 
See ede aed 
= t eee — <1, 
In(l+x)=x 5) 3 a for -l<x<l 
2 3 
(14x) =14nx + n(n D> +nn 1(n ae for —w0 <x <0. 


4.7 EVALUATION OF LIMITS 


These can all be derived by straightforward application of Taylor’s theorem to 
the expansion of a function about x = 0. 


4.7 Evaluation of limits 


The idea of the limit of a function f(x) as x approaches a value a is fairly intuitive, 
though a strict definition exists and is stated below. In many cases the limit of 
the function as x approaches a will be simply the value f(a), but sometimes this 
is not so. Firstly, the function may be undefined at x = a, as, for example, when 


sin x 


f(x) = —, 


x 


which takes the value 0/0 at x = 0. However, the limit as x approaches zero 
does exist and can be evaluated as unity using l’H6pital’s rule below. Another 
possibility is that even if f(x) is defined at x = a its value may not be equal to the 
limiting value lim,-_,, f(x). This can occur for a discontinuous function at a point 
of discontinuity. The strict definition of a limit is that if lim... f(x) = | then 
for any number e however small, it must be possible to find a number yn such that 
|f(x)—l| < € whenever |x—a| < yn. In other words, as x becomes arbitrarily close to 
a, f(x) becomes arbitrarily close to its limit, /. To remove any ambiguity, it should 
be stated that, in general, the number 7 will depend on both e and the form of f(x). 
The following observations are often useful in finding the limit of a function. 


(i) A limit may be +oo. For example as x > 0, 1/x? > o. 

(ii) A limit may be approached from below or above and the value may be 
different in each case. For example consider the function f(x) = tanx. As x tends 
to 2/2 from below f(x) — 00, but if the limit is approached from above then 
f(x) ~ —o. Another way of writing this is 

lim tan x = 00, lim tan x = —oo. 
X57 x5 

(iii) It may ease the evaluation of limits if the function under consideration is 
split into a sum, product or quotient. Provided that in each case a limit exists, the 
rules for evaluating such limits are as follows. 


(a) lim {f(x) + g(x)} = lim f(x) + lim g(x). 
(b) lim {f(2o)g(x)} = lim f(x) lim g(x). 


hi Xa : 
(c) lim Sx) = lim, a SC) provided that 
xoa g(x) lim, a g(x) 
the numerator and denominator are 


not both equal to zero or infinity. 


Examples of cases (a)-(c) are discussed below. 
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> Evaluate the limits 
sin x 


lim(x? + 2x3), lim(x cos x), lim : 
x1 x0 xon/2 xX 


Using (a) above, 
lim(x* + 2x°) = lim x? + lim 2x° = 3. 


x1 

Using (b), 

lim(x cos x) = limx limcosx =0x 1 =0. 

x x30 x 
Using (c), 

: i lim, 7/2 sin x 1 
lim ae se RIE .< 
xon/2 xX lim, 2/2 x m/2 


(iv) Limits of functions of x that contain exponents that themselves depend on 
x can often be found by taking logarithms. 


> Evaluate the limit 


Let us define 


a\* 
es) 


and consider the logarithm of the required limit, i.e. 


2: 
lim In y = lim [sin (: = 5)| 
Using the Maclaurin series for In(1 + x) given in subsection 4.6.3, we can expand the 
logarithm as a series and obtain 


2 4 
Jim In y = lim le (-S-st)] sagt 


Therefore, since lim,_,., ln y = —a? it follows that lim,_,.. y = exp(—a’). < 


(v) LHOpital’s rule may be used; it is an extension of (iii)(c) above. In cases 
where both numerator and denominator are zero or both are infinite, further 
consideration of the limit must follow. Let us first consider lim, a f(x)/g(x), 
where f(a) = g(a) = 0. Expanding the numerator and denominator as Taylor 
series we obtain 

fx) — fa@t+o—af'(a +le—a’/2f"(@ +-- 
a(x) — g(a) + (x — a)g’(a) + [(x — a)?/2"g"(a) + 
However, f(a) = g(a) =0 so 


f(x) — fia +(x —a)/2"f"(a) +++ 
a(x) g(a) + [(x —a)/2!]2"(a) +> 
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Therefore we find 
. f(x) _ f'@) 
lim —— = A 
xa g(x) — g’(a) 


provided f’(a) and g’(a) are not themselves both equal to zero. If, however, 
f'(a) and g'(a) are both zero then the same process can be applied to the ratio 
F'(x)/g'(x) to yield 

imi 7 

xa g(x) g"(a) 


provided that at least one of f”(a) and g”(a) is non-zero. If the original limit does 
exist then it can be found by repeating the process as many times as is necessary 
for the ratio of corresponding nth derivatives not to be of the indeterminate form 
0/0, i.e. 
fag Os TM), 
wea g(x) g(a) 


> Evaluate the limit 


We first note that if x = 0, both numerator and denominator are zero. Thus we apply 
l'H6pital’s rule: differentiating, we obtain 


lim(sin x/x)= lim(cos x/lj)=1.< 


So far we have only considered the case where f(a) = g(a) = 0. For the case 
where f(a) = g(a) = © we may still apply l’H6pital’s rule by writing 


fo) _ 4 Vets) 


im ——~ = ays 

xa g(x) xa 1/ f(x) 
which is now of the form 0/0 at x = a. Note also that ’H6pital’s rule is still 
valid for finding limits as x > 00, 1.e. when a = oo. This is easily shown by letting 
y =1/x as follows: 


oS) Lg Ay) 


gee BU ra (1/9) 
-!RFum 
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Summary of methods for evaluating limits 


To find the limit of a continuous function f(x) at a point x = a, simply substitute 
the value a into the function noting that 2 = 0 and that | = o. The only 
difficulty occurs when either of the expressions 3 or © results. In this case 
differentiate top and bottom and try again. Continue differentiating until the top 
and bottom limits are no longer both zero or both infinity. If the undetermined 


form 0 x oo occurs then it can always be rewritten as q or =. 


4.8 Exercises 


4.1 Sum the even numbers between 1000 and 2000 inclusive. 

4.2 If you invest £1000 on the first day of each year, and interest is paid at 5% on 
your balance at the end of each year, how much money do you have after 25 
years? 

43 How does the convergence of the series 


“(n—r)! 
De n! 


n=r 


depend on the integer r? 


44 Show that for testing the convergence of the series 
xtytxe+y4exr ty te, 


where 0 < x < y < 1, the D’Alembert ratio test fails but the Cauchy root test is 
successful. 


4.5 Find the sum Sy of the first N terms of the following series, and hence determine 
whether the series are convergent, divergent or oscillatory: 


oa) ca] © (¢__1)ntl 
(a) on (=). © er, @ pO 


n=1 n=0 n=1 


4.6 By grouping and rearranging terms of the absolutely convergent series 


show that 


47 Use the difference method to sum the series 


4.8 EXERCISES 


4.8 


4.9 


4.10 


4.11 


4.12 


4.13 


4.14 


The N +1 complex numbers @, are given by @, = exp(2zim/N), for m = 
0,1,2,...,N 


(a) Evaluate the following: 


N N N 
Lia GS me, Gi) Saga” 


m=0 m=0 m=0 


(b) Use these results to evaluate: 


2n 4nm rs m 
(i) > os (=) —c¢ (4) , (i) ae si ( 3 ) 
Prove that 
sin aU + lo 
cos @ + cos(@ + a) +--+ +cos(0 + na) = —*——— cos(0 + 3nd). 
sin 5 ta 


Determine whether the following series converge (@ and p are positive real 
numbers): 


oO 


*\ 2sinn0 2 ea | 
) Do nat Wy (b) rs (<) 2nt/2” 


(=1)"(n? +11? n? 
oo ninn > ©) a 


@ DA & Loin, © De 
n=1 n=1 n=1 
> e”™ (e) > (la n) 
n=1 n=2 


Determine whether the following series are absolutely convergent, convergent or 
oscillatory: 


(= ~. (—1)"(2n + 1) (=1)"|x|" 
oS wo pe | yo 


n=1 n=1 n=0 


( 1)" @. (—1)"2" 
(d) Dera (e) a nz * 


n=1 


Obtain the positive values of x for which the following series converges: 


io) 
xn/2 en 
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4.15 


4.16 


4.17 


4.18 


4.19 


Prove that 


& n’ +(—1)" 
2m a | 


is absolutely convergent for r = 2, but only conditionally convergent for r = 1. 


An extension to the proof of the integral test (subsection 4.3.2) shows that, if f(x) 
is positive, continuous and monotonically decreasing, for x > 1, and the series 
fd)+f(2) +--+ is convergent, then its sum does not exceed f(1) + L, where L 


is the integral 
| f(x) dx. 
1 


Use this result to show that the sum ¢(p) of the Riemann zeta series > n-?, with 
p> 1, is not greater than p/(p — 1). 


Demonstrate that rearranging the order of its terms can make a condition- 
ally convergent series converge to a different limit by considering the series 
Yo(-1)"*'n7! = In2 = 0.693. Rearrange the series as 


oat 1 1 1 1 1 1 1 1 1 
S=qt3-atsty-gtotpostat 


and group each set of three successive terms. Show that the series can then be 
written 


which is convergent (by comparison with )>n~?) and contains only positive 
terms. Evaluate the first of these and hence deduce that S is not equal to In 2. 


Illustrate result (iv) of section 4.4, concerning Cauchy products, by considering 
the double summation 


ioe) n 1 
i De Serger: 


n=1 r=1 


By examining the points in the nr-plane over which the double summation is to 
be carried out, show that S can be written as 


SS 2 aga 


n=r r=1 


Deduce that S < 3. 


A Fabry-Pérot interferometer consists of two parallel heavily silvered glass plates; 
light enters normally to the plates, and undergoes repeated reflections between 
them, with a small transmitted fraction emerging at each reflection. Find the 
intensity of the emerging wave, |B|?, where 


B=A(1—-r) > re", 


n=0 


with r and ¢ real. 
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4.20 


4.21 


4.22 


4.23 


4.24 


4.25 


4.26 


4.27 


4.28 


Identify the series 
2 (—1)"*! x2" 


Qn—1!’ 


n=1 


and then, by integration and differentiation, deduce the values S of the following 
series: 


ay! rn 2 ( 1)"t'n 


Cpa a? Oe aren 


n=1 n=1 
ro) 


ee (—1)"(n4+ 1) 
ose Faeyr Oey 


Starting from the Maclaurin series for cos x, show that 


4 
(cosxy?= tt Bye, 


Deduce the first three terms in the Maclaurin series for tan x. 
Find the Maclaurin series for: 


(a) n(**), (b) G2 +4), (ce) sin’ x. 


1-x 
Writing the nth derivative of f(x) = sinh7! x as 


P,,(x) 
(n) Pa n 
LOO) = Gey 
where P,,(x) is a polynomial (of order n — 1), show that the P,(x) satisfy the 
recurrence relation 


Prsi(x) = (1+ x°)Pi(x) — (20 — 1)xP, (x). 


Hence generate the coefficients necessary to express sinh! x as a Maclaurin series 
up to terms in x°. 

Find the first three non-zero terms in the Maclaurin series for the following 
functions: 


(a)? +9)'7,  (b) Inf(2+x)*], (c) exp(sin x), 
(d) In(cos x), (e) exp[-(x —a)~*], (f) tan! x. 


By using the logarithmic series, prove that if a and b are positive and nearly 
equal then 
a 2(a—b) 

In-~ : 

"pb a+b 
Show that the error in this approximation is about 2(a— b)?/[3(a + b)’]. 
Determine whether the following functions f(x) are (i) continuous, and (ii) 
differentiable at x = 0: 


) f(x) = exp(—|xl); 
a f(x) = (1 —cos x)/x? for x #0, 10) = 3 
(c) f(x) =xsin(1/x) for x #0, f(0) = 
(d) f(x) = [4—x?], where [y] denotes if integer part of y. 


Find the limit as x > 0 of L/1 + x" —./1 — x"]/x", in which m and n are positive 
integers. 
Evaluate the following limits: 
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4.29 


4.30 


4.31 


4.32 


4.33 


4.34 


sin 3x . tanx —tanhx 
im — ‘ (b) lim ——————_,, 
x0 sinh x x>0 sinhx—x 
tanx—x . cosecx  sinhx 
(c) lim ———,  (d) lim po Sa 
x0 cosx — 1 x0 x Bs 


Find the limits of the following functions: 


+x? —5x—2 
(a) Oe Th dep asx >0,x > 00 and x > 2; 


sin x — x coshx 


: asx > 0; 


(b) sinh x — x 


n/2 gn 45 

(c) i, (et) dy, asx > 0. 
x y 

Use Taylor expansions to three terms to find approximations to (a) +,/17, and 

(b) 3./26. 

Using a first-order Taylor expansion about x = Xo, show that a better approxi- 

mation than xo to the solution of the equation 


f(x) = sinx + tanx = 2 
is given by x = xX) + 6, where 


2 — f (xo) 


6= SS 
COS Xo + SeC* Xo 


(a) Use this procedure twice to find the solution of f(x) = 2 to six significant 
figures, given that it is close to x = 0.9. 

(b) Use the result in (a) to deduce, to the same degree of accuracy, one solution 
of the quartic equation 


yt —4y3+4y? +4y-—4=0. 


lim u cosec x ee 
x0 | x3 7 x 6 . 


In quantum theory, a system of oscillators, each of fundamental frequency v and 
interacting at temperature T, has an average energy E given by 


00. —nx 
a ) nhyve""™* 
= nv 
E = ‘ eax 
oreo 


where x = hv/kT, h and k being the Planck and Boltzmann constants, respec- 
tively. Prove that both series converge, evaluate their sums, and show that at high 
temperatures E ~ kT, whilst at low temperatures E ~ hv exp(—hv/kT). 

In a very simple model of a crystal, point-like atomic ions are regularly spaced 
along an infinite one-dimensional row with spacing R. Alternate ions carry equal 
and opposite charges +e. The potential energy of the ith ion in the electric field 
due to another ion, the jth, is 


Evaluate 


G4) 
4neorij > 
where q;, qj are the charges on the ions and rj; is the distance between them. 


Write down a series giving the total contribution V; of the ith ion to the overall 
potential energy. Show that the series converges, and, if V; is written as 


ae? 


Vi=7—, 
4neoR 
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4.35 


4.36 


4.11 


4.13 


find a closed-form expression for «, the Madelung constant for this (unrealistic) 
lattice. 

One of the factors contributing to the high relative permittivity of water to static 
electric fields is the permanent electric dipole moment, p, of the water molecule. 
In an external field E the dipoles tend to line up with the field, but they do not 
do so completely because of thermal agitation corresponding to the temperature, 
T, of the water. A classical (non-quantum) calculation using the Boltzmann 
distribution shows that the average polarisability per molecule, «, is given by 


a= 2 (cothx —x}), 


where x = pE/(kT) and k is the Boltzmann constant. 

At ordinary temperatures, even with high field strengths (10* Vm~! or more), 
x < 1. By making suitable series expansions of the hyperbolic functions involved, 
show that « = p?/(3kT) to an accuracy of about one part in 15x7?. 
In quantum theory, a certain method (the Born approximation) gives the (so- 
called) amplitude f(0) for the scattering of a particle of mass m through an angle 
@ by a uniform potential well of depth Vo and radius b (i.e. the potential energy 
of the particle is —Vo within a sphere of radius b and zero elsewhere) as 


2mVo 
{(0) = 


Here h is the Planck constant divided by 27, the energy of the particle is h?k? /(2m) 
and K is 2k sin(@/2). 
Use I’H6pital’s rule to evaluate the amplitude at low energies, ic. when k and 
hence K tend to zero, and so determine the low-energy total cross-section. 
[Note: the differential cross-section is given by |f(0)|? and the total cross- 
section by the integral of this over all solid angles, i.e. 27 a |f(@)/? sin @ dé.] 


1 


(sin Kb — KbcosKb). 


4.9 Hints and answers 
Write as 2(57 2? n — 374, n) = 751500. 


Divergent for r ‘< 1; convergent for r > 2. 

(a) In(N + 1), divergent; (b) +[1 —(—2)"], oscillates infinitely; (c) Add 3Sy to the 
Sy series; %[1 —(—3)-%] + 3N(—3)-*—, convergent to 4. 
Write the nth term as the difference between two consecutive values of a partial- 
fraction function of n. The sum equals (1 —N~). 


Sum the geometric series with rth term exp[i(@ + ra)]. Its real part is 
{cos 0 — cos [(n + 1)u + 0] — cos(0 — «) + cos(@ + na)} /4 sin?(«/2), 
which can be reduced to the given answer. 


(a) —1 < x < 1; (b) all x except x = (2n+ 1)m/2; (c) x < —1; (d) x < 0; (e) 
always divergent. Clearly divergent for x > —1. For —X = x < —1, consider 


1 


Dee (In Mx)*" 


k=1 n=M,_14+1 


where In M;, =k and note that M, — My_; = e~'(e — 1)M,; hence show that the 
series diverges. 

(a) Absolutely convergent, compare with exercise 4.10(b). (b) Oscillates finitely. 
(c) Absolutely convergent for all x. (d) Absolutely convergent; use partial frac- 
tions. (e) Oscillates infinitely. 
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SERIES AND LIMITS 


4.15 


4.17 
4.19 
4.21 


4.23 
4.25 
4.27 
4.29 
4.31 


4.33 


4.35 


Divide the series into two series, n odd and n even. For r = 2 both are absolutely 
convergent, by comparison with )>n~?. For r = 1 neither series is convergent, 
by comparison with )> n~!. However, the sum of the two is convergent, by the 
alternating sign test or by showing that the terms cancel in pairs. 

The first term has value 0.833 and all other terms are positive. 

|AP( —r)?/(1 +r? — 2rcos @). 

Use the binomial expansion and collect terms up to x*. Integrate both sides of 
the displayed equation. tanx = x + x°/3 4+ 2x°/154+->-. 

For example, Ps(x) = 24x* — 72x? + 9. sinh7! x = x — x3/6 4+ 3x°/40—---. 

Set a= D+ 6 and b = D—6 and use the expansion for In(1 + 6/D). 

The limit is 0 for m>n, 1 for m=n, and o for m <n. 

(a) —3, 4, 00; (b) —4; (c) -14+-2/n. 

(a) First approximation 0.886452; second approximation 0.886287. (b) Set y = 
sinx and re-express f(x) = 2 as a polynomial equation. y = sin(0.886287) = 
0.774730. 

If S(x) = 30.) e-™ evaluate S(x) and consider dS(x)/dx. 

E = hy[exp(hv/kT) — 1]!. 


‘ pace SP fA «xP 
Th : BD ieee eh aay 
e series expansion is = ( eae ) 
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5 


Partial differentiation 


In chapter 2, we discussed functions f of only one variable x, which were usually 
written f(x). Certain constants and parameters may also have appeared in the 
definition of f, e.g. f(x) = ax+2 contains the constant 2 and the parameter a, but 
only x was considered as a variable and only the derivatives f(x) = d"f /dx" 
were defined. 

However, we may equally well consider functions that depend on more than one 
variable, e.g. the function f(x, y) = x? + 3xy, which depends on the two variables 
x and y. For any pair of values x, y, the function f(x, y) has a well-defined value, 
e.g. f(2,3) = 22. This notion can clearly be extended to functions dependent on 
more than two variables. For the n-variable case, we write f(x1,X2,...,Xn) for 
a function that depends on the variables x1,x2,...,Xn. When n = 2, x; and x2 
correspond to the variables x and y used above. 

Functions of one variable, like f(x), can be represented by a graph on a 
plane sheet of paper, and it is apparent that functions of two variables can, 
with little effort, be represented by a surface in three-dimensional space. Thus, 
we may also picture f(x,y) as describing the variation of height with position 
in a mountainous landscape. Functions of many variables, however, are usually 
very difficult to visualise and so the preliminary discussion in this chapter will 
concentrate on functions of just two variables. 


5.1 Definition of the partial derivative 


It is clear that a function f(x,y) of two variables will have a gradient in all 
directions in the xy-plane. A general expression for this rate of change can be 
found and will be discussed in the next section. However, we first consider the 
simpler case of finding the rate of change of f(x,y) in the positive x- and y- 
directions. These rates of change are called the partial derivatives with respect 
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to x and y respectively, and they are extremely important in a wide range of 
physical applications. 

For a function of two variables f(x, y) we may define the derivative with respect 
to x, for example, by saying that it is that for a one-variable function when y is 
held fixed and treated as a constant. To signify that a derivative is with respect 
to x, but at the same time to recognize that a derivative with respect to y also 
exists, the former is denoted by 0f/0x and is the partial derivative of f(x,y) with 
respect to x. Similarly, the partial derivative of f with respect to y is denoted by 
of /Oy. 

To define formally the partial derivative of f(x,y) with respect to x, we have 

of . ‘(x + Ax, y) — f(x, 

at a (51) 
provided that the limit exists. This is much the same as for the derivative of a 
one-variable function. The other partial derivative of f(x,y) is similarly defined 
as a limit (provided it exists): 

af = tim Loe tA = fey) (5.2) 

dy Ay Ay 

It is common practice in connection with partial derivatives of functions 
involving more than one variable to indicate those variables that are held constant 
by writing them as subscripts to the derivative symbol. Thus, the partial derivatives 
defined in (5.1) and (5.2) would be written respectively as 


of of 
a) pus (F) 


In this form, the subscript shows explicitly which variable is to be kept constant. 
A more compact notation for these partial derivatives is f, and f,. However, it is 
extremely important when using partial derivatives to remember which variables 
are being held constant and it is wise to write out the partial derivative in explicit 
form if there is any possibility of confusion. 

The extension of the definitions (5.1), (5.2) to the general n-variable case is 
straightforward and can be written formally as 


Of (X1,X2,-+-5Xn) lim Lf (1, X2,+++, Xi FAX: Xn) — f(X1 X25 +0 Xin Xn) 
OX; Axi>0 AX; , 


provided that the limit exists. 
Just as for one-variable functions, second (and higher) partial derivatives may 
be defined in a similar way. For a two-variable function f(x, y) they are 


a (of\ Pf Of OTN ed 
5. \ aT = 55 HS avekevio oe z= Sy, 
Ox \ 0x Ox y \ey oy 

0 (of\  &f ay 0 (of\  &f = 
ox \éy éxdy —-*” dy \ dx aéyax ~*~ 
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5.2 THE TOTAL DIFFERENTIAL AND TOTAL DERIVATIVE 


Only three of the second derivatives are independent since the relation 


ef Of 


éxdy  dyéx’ 
is always obeyed, provided that the second partial derivatives are continuous 
at the point in question. This relation often proves useful as a labour-saving 
device when evaluating second partial derivatives. It can also be shown that for 
a function of n variables, f(x,,x2,...,Xn), under the same conditions, 


> Find the first and second partial derivatives of the function 


f(x,y) = 2x3y* + y°. 


The first partial derivatives are 


0 0 
of = 6x7y’, of = 43y + 3y 
Ox oy 
and the second partial derivatives are 
Of of of Of 
— = 12xy? — =4x7 +6 = 12x? = 12x°y 
a 2 a he + 6y, axdy tr Br x’y, 


the last two being equal, as expected. < 


5.2 The total differential and total derivative 


Having defined the (first) partial derivatives of a function f(x, y), which give the 
rate of change of f along the positive x- and y-axes, we consider next the rate of 
change of f(x,y) in an arbitrary direction. Suppose that we make simultaneous 
small changes Ax in x and Ay in y and that, as a result, f changes to f + Af. 
Then we must have 


Af = f(x +Ax,y + Ay) — f(xy) 
= f(x +Ax,y + Ay) — f(x,y + Ay) + f(x,y + Ay) — f(xy) 


_ [f+ Axy t+ Ay)— feoy +4y)] a [Foy + Ay) — F069) 4 
Ax Ay 2 


(5.3) 


In the last line we note that the quantities in brackets are very similar to those 

involved in the definitions of partial derivatives (5.1), (5.2). For them to be strictly 

equal to the partial derivatives, Ax and Ay would need to be infinitesimally small. 

But even for finite (but not too large) Ax and Ay the approximate formula 
Kp GHD) 5 ose CAO) ths 


Ox oy " 


(5.4) 
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can be obtained. It will be noticed that the first bracket in (5.3) actually approxi- 
mates to 0f(x,y + Ay)/éx but that this has been replaced by 0f(x, y)/0x in (5.4). 
This approximation clearly has the same degree of validity as that which replaces 
the bracket by the partial derivative. 

How valid an approximation (5.4) is to (5.3) depends not only on how small 
Ax and Ay are but also on the magnitudes of higher partial derivatives; this is 
discussed further in section 5.7 in the context of Taylor series for functions of 
more than one variable. Nevertheless, letting the small changes Ax and Ay in 
(5.4) become infinitesimal, we can define the total differential df of the function 
f(x,y), without any approximation, as 


of 


df = dx + shay 
Ox 


y. (5.5) 
Equation (5.5) can be extended to the case of a function of n variables, 
F(X1, 2, +++ Xn); 
._ of of 
df => ayo i+ at +. of 


of 


Ox, ——dXp. (5.6) 


> Find the total differential of the function f(x,y) = yexp(x + y). 


Evaluating the first partial derivatives, we find 


of of 
ae yexp(x + y), a exp(x + y) + yexp(x + y). 


Applying (5.5), we then find that the total differential is given by 
df = [yexp(x + y)]dx + [(1 + y)exp(x + y)]dy. < 


In some situations, despite the fact that several variables x;, i = 1,2,...,n, 
appear to be involved, effectively only one of them is. This occurs if there are 
subsidiary relationships constraining all the x; to have values dependent on the 
value of one of them, say x;. These relationships may be represented by equations 
that are typically of the form 


xp =xAx1), i= 2,3,...57. (5.7) 


In principle f can then be expressed as a function of x; alone by substituting 
from (5.7) for x2, x3,...,Xn, and then the total derivative (or simply the derivative) 
of f with respect to x; is obtained by ordinary differentiation. 

Alternatively, (5.6) can be used to give 


df of Of \ dx Of \ dxXn 

dx; Ox (= dx + OX, J dx, Ce) 
It should be noted that the LHS of this equation is the total derivative df /dx,, 
whilst the partial derivative 0f/0x, forms only a part of the RHS. In evaluating 
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5.3 EXACT AND INEXACT DIFFERENTIALS 


this partial derivative account must be taken only of explicit appearances of x; in 
the function f, and no allowance must be made for the knowledge that changing 
x1 necessarily changes x2, X3,...,Xn. The contribution from these latter changes is 
precisely that of the remaining terms on the RHS of (5.8). Naturally, what has 
been shown using x; in the above argument applies equally well to any other of 
the x;, with the appropriate consequent changes. 


> Find the total derivative of f(x,y) = x2 + 3xy with respect to x, given that y = sin | x. 


We can see immediately that 


of _ of dy 1 
ot. Bye * ae ee 
and so, using (5.8) with x} = x and x. = y, 
df 1 
a: OST OY ON re eee 
3x 
4 
=2x+3sin ay ae 


Obviously the same expression would have resulted if we had substituted for y from the 
start, but the above method often produces results with reduced calculation, particularly 
in more complicated examples. < 


5.3. Exact and inexact differentials 


In the last section we discussed how to find the total differential of a function, ie. 
its infinitesimal change in an arbitrary direction, in terms of its gradients 0f /0x 
and df /dy in the x- and y- directions (see (5.5)). Sometimes, however, we wish 
to reverse the process and find the function f that differentiates to give a known 
differential. Usually, finding such functions relies on inspection and experience. 

As an example, it is easy to see that the function whose differential is df = 
x dy + ydx is simply f(x,y) = xy +c, where c is a constant. Differentials such as 
this, which integrate directly, are called exact differentials, whereas those that do 
not are inexact differentials. For example, x dy + 3y dx is not the straightforward 
differential of any function (see below). Inexact differentials can be made exact, 
however, by multiplying through by a suitable function called an integrating 
factor. This is discussed further in subsection 14.2.3. 


> Show that the differential x dy + 3y dx is inexact. 


On the one hand, if we integrate with respect to x we conclude that f(x,y) = 3xy + g(y), 
where g(y) is any function of y. On the other hand, if we integrate with respect to y we 
conclude that f(x,y) = xy + h(x) where h(x) is any function of x. These conclusions are 
inconsistent for any and every choice of g(y) and h(x), and therefore the differential is 
inexact. 4 


It is naturally of interest to investigate which properties of a differential make 
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it exact. Consider the general differential containing two variables, 
df = A(x, y) dx + B(x, y) dy. 
We see that 


5 ar 


=Atsy = Boxy) 


id} 
s 


0 


2 


and, using the property fyy = fyx, we therefore require 


dA 0B 
rs (5.9) 


This is in fact both a necessary and a sufficient condition for the differential to 
be exact. 


> Using (5.9) show that x dy + 3y dx is inexact. 


In the above notation, A(x, y) = 3y and B(x, y) = x and so 
0A 0B 
0A _ <9 


oy > Ox 


As these are not equal it follows that the differential is inexact. < 


Determining whether a differential containing many variable x1,x2,...,Xn is 
exact is a simple extension of the above. A differential containing many variables 
can be written in general as 


n 
df = » Bi(X1,X2, +++) Xn) AX; 
i=1 


and will be exact if 


; Ao: 
ax, = ia for all pairs i, j. (5.10) 


There will be $n(n — 1) such relationships to be satisfied. 


> Show that 
(y+2)dx+xdy+xdz 


is an exact differential. 


In this case, g1(x,y,z) = y+2Z, g(x, y,z) = X, g3(x, y,z) = x and hence dg,/dy = 1 = 
0g/0x, 0g3/0x = 1 = 0g,/02, 0g2/0z = 0 = 0g3/dy; therefore, from (5.10), the differential 
is exact. As mentioned above, it is sometimes possible to show that a differential is exact 
simply by finding by inspection the function from which it originates. In this example, it 
can be seen easily that f(x, y,z) =x(y+z)+c < 
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5.4 USEFUL THEOREMS OF PARTIAL DIFFERENTIATION 


5.4 Useful theorems of partial differentiation 


So far our discussion has centred on a function f(x, y) dependent on two variables, 
x and y. Equally, however, we could have expressed x as a function of f and y, 
or y as a function of f and x. To emphasise the point that all the variables are 
of equal standing, we now replace f by z. This does not imply that x, y and z 
are coordinate positions (though they might be). Since x is a function of y and z, 
it follows that 


a A 
dx = (=) dy + (S) dz (5.11) 
oy), éz/, 
and similarly, since y = y(x,z), 
A A 
i= (2) dx+ (2) dz. (5.12) 
Ox 3 OZ 7 


We may now substitute (5.12) into (5.11) to obtain 


= Ox oy ’ Ox oy Ox 
. (2) (2.+|(@),@+@ J+ (5.13) 


Now if we hold z constant, so that dz = 0, we obtain the reciprocity relation 


ax\ — fay\" 
dy), \éx],’ 


which holds provided both partial derivatives exist and neither is equal to zero. 
Note, further, that this relationship only holds when the variable being kept 
constant, in this case z, is the same on both sides of the equation. 

Alternatively we can put dx = 0 in (5.13). Then the contents of the square 
brackets also equal zero, and we obtain the cyclic relation 


oy Oz Ox\ _ 1 
6z),\ Ox), \Oy/, : 


which holds unless any of the derivatives vanish. In deriving this result we have 
used the reciprocity relation to replace (6x/dz)," by (@z/0x),. 


5.5 The chain rule 


So far we have discussed the differentiation of a function f(x,y) with respect to 
its variables x and y. We now consider the case where x and y are themselves 
functions of another variable, say u. If we wish to find the derivative df/du, 
we could simply substitute in f(x,y) the expressions for x(u) and y(u) and then 
differentiate the resulting function of u. Such substitution will quickly give the 
desired answer in simple cases, but in more complicated examples it is easier to 
make use of the total differentials described in the previous section. 
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From equation (5.5) the total differential of f(x,y) is given by 
. of of 
df = 5 dx + ay 


0) 


but we now note that by using the formal device of dividing through by du this 
immediately implies 
df _ofdx | of dy 


meee Be cismigu Bild 5.14 
du Oxdu dydw CY 


which is called the chain rule for partial differentiation. This expression provides 
a direct method for calculating the total derivative of f with respect to u and is 
particularly useful when an equation is expressed in a parametric form. 


Given that x(u) = 1+ au and y(u) = bu?, find the rate of change of f(x,y) = xe~’ with 
respect to u. 


As discussed above, this problem could be addressed by substituting for x and y to obtain 
f as a function only of u and then differentiating with respect to u. However, using (5.14) 
directly we obtain 


d ; 
af = (ea + (—xe)3bu", 
du 
which on substituting for x and y gives 
df 


=e" (a 3bu> — 3bau?). < 
du 


Equation (5.14) is an example of the chain rule for a function of two variables 
each of which depends on a single variable. The chain rule may be extended to 
functions of many variables, each of which is itself a function of a variable u, i.e. 


f (X1, 2, X3,-..,Xn), With x; = x;(u). In this case the chain rule gives 
df — y Of dxi _ a dx, of dxy (ee Of dXn (5.15) 
du = Ox; du Ox; du Oxy du Ox, du 


5.6 Change of variables 


It is sometimes necessary or desirable to make a change of variables during the 
course of an analysis, and consequently to have to change an equation expressed 
in one set of variables into an equation using another set. The same situation arises 
if a function f depends on one set of variables x;, so that f = f (x1, X2,...,Xn) but 
the x; are themselves functions of a further set of variables u; and given by the 
equations 


Xi = Xi(Uy, U2,--- Um). (5.16) 
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Figure 5.1 The relationship between Cartesian and plane polar coordinates. 


For each different value of i, x; will be a different function of the uj. In this case 
the chain rule (5.15) becomes 


Of _ ". Of 6x; _ 
Bu, Sea j=1,2,....m, (5.17) 


and is said to express a change of variables. In general the number of variables 
in each set need not be equal, i.e. m need not equal n, but if both the x; and the 
u; are sets of independent variables then m = n. 


> Plane polar coordinates, p and @, and Cartesian coordinates, x and y, are related by the 
expressions 


x=pcos?,  y=psing, 


as can be seen from figure 5.1. An arbitrary function f(x,y) can be re-expressed as a 
function g(p, p). Transform the expression 


Of OF 
Ox? = dy? 


into one in p and ¢. 


We first note that p? = x? + y?, @ = tan7!(y/x). We can now write down the four partial 
derivatives 


il ~ cos op —y/*?) sin ? 
Ox (x2 + y*)'/2 » as as (y/x)? p > 
Op _ y Se 6p 1/x  _ coso 
dy Gee By TERPS 
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Thus, from (5.17), we may write 


sing 0 0 ; 0 cosd 0 
p op oy opp Og 


0 0 
ag cos os, 


Now it is only a matter of writing 


®f_ 9 (a)_a(2), 
6x2 ax \dx) dx \ x 
6 sing 0 ) ( 0 sing 0 ) 
cos cos _ = 
( °p p30) \*a p20) * 
= (coro - at) (cos 98 _ wt) 
op p oo op p 0 
2,0 | 2cospsind A 2cospsind 67g 


g 
p dg p dpep 
sin’ pd dg . sin’ Og 
p Op pe og? 


and a similar expression for 67f /dy’, 


o 6 cosd 0 6 cosd 0 
ay? (snoz5+ ar (sinoz, + reas 
> ,@g 2cosPsind dg £ 2cosPsind dg 
op? i oo p Oop 
cos*p dg cos’ 0*g 
p op. pr ag? 


When these two expressions are added together the change of variables is complete and 
we obtain 


5.7 Taylor’s theorem for many-variable functions 


We have already introduced Taylor’s theorem for a function f(x) of one variable, 
in section 4.6. In an analogous way, the Taylor expansion of a function f(x, y) of 
two variables is given by 


. < hig of of 
f(x,y) = F(X0, yo) + 3x4 + ay 
De aie Ae as Oe 
tor 3x2 Ax) Br + a7 dy) fine, S18) 


where Ax = x — Xp and Ay = y — yo, and all the derivatives are to be evaluated 


at (xo, Yo): 


160 


5.7 TAYLOR’S THEOREM FOR MANY-VARIABLE FUNCTIONS 


> Find the Taylor expansion, up to quadratic terms in x —2 and y —3, of f(x,y) = yexpxy 
about the point x =2, y =3. 


We first evaluate the required partial derivatives of the function, i.e. 


5 7 
ah = y? exp xy, of = eXpXxy + xy exp xy, 
Ox oy 

OP af P 

Fad = Y EXP XY, ay = 2x exp xy + x*yexpxy, 

of 


<_ = 2yexpxy + xy” exp xy. 
Oxdy 


Using (5.18), the Taylor expansion of a two-variable function, we find 
f(xy) ~ &{3 + 9x — 2) + 19-3) 


+ (217! [27(x — 2)? + 48(x — 2)(y — 3) + 16(y — 3)7] \. < 


It will be noticed that the terms in (5.18) containing first derivatives can be 
written as 


Ax 4 ay) (a» . FAYS) fly) 


jad) 


where both sides of this relation should be evaluated at the point (xo, yo). Similarly 
the terms in (5.18) containing second derivatives can be written as 


A2¢ q2¢ 
1 0 Fax +224 
x 


ef 1 o Ones 
axdy+ iar] = 5 (are tars) fo) 


AxA 2 
ey. - (5.19) 


where it is understood that the partial derivatives resulting from squaring the 
expression in parentheses act only on f(x,y) and its derivatives, and not on Ax 
or Ay; again both sides of (5.19) should be evaluated at (xo, yo). It can be shown 
that the higher-order terms of the Taylor expansion of f(x, y) can be written in 
an analogous way, and that we may write the full Taylor series as 


: “1 7 ae 
foan= 5 (are tare) fers) 


n=0 


X00 


where, as indicated, all the terms on the RHS are to be evaluated at (Xo, yo). 
The most general form of Taylor’s theorem, for a function f(x1,X2,...,X,) of n 
variables, is a simple extension of the above. Although it is not necessary to do 
so, we may think of the x; as coordinates in n-dimensional space and write the 
function as f(x), where x is a vector from the origin to (x1, X2,...,X,). Taylor’s 


161 


PARTIAL DIFFERENTIATION 


theorem then becomes 


of orf 
f(x) = 00) + 9 a5, oe +> ey "es a (5.20) 


where Ax; = x; — x; and the partial derivatives are evaluated at (x1), X2),.--, Xin): 
For completeness, we note that in this case the full Taylor series can be written 
in the form 


ee) 


£00) = [AK VY") ay 


n=0 


where V is the vector differential operator del, to be discussed in chapter 10. 


5.8 Stationary values of many-variable functions 


The idea of the stationary points of a function of just one variable has already 
been discussed in subsection 2.1.8. We recall that the function f(x) has a stationary 
point at x = Xo if its gradient df/dx is zero at that point. A function may have 
any number of stationary points, and their nature, i.e. whether they are maxima, 
minima or stationary points of inflection, is determined by the value of the second 
derivative at the point. A stationary point is 


(i) a minimum if d?f /dx? > 0; 
(ii) a maximum if d?f /dx? <0; 
(iii) a stationary point of inflection if d?f/dx? = 0 and changes sign through 
the point. 


We now consider the stationary points of functions of more than one variable; 
we will see that partial differential analysis is ideally suited to the determination 
of the position and nature of such points. It is helpful to consider first the case 
of a function of just two variables but, even in this case, the general situation 
is more complex than that for a function of one variable, as can be seen from 
figure 5.2. 

This figure shows part of a three-dimensional model of a function f(x,y). At 
positions P and B there are a peak and a bowl respectively or, more mathemati- 
cally, a local maximum and a local minimum. At position S the gradient in any 
direction is zero but the situation is complicated, since a section parallel to the 
plane x = 0 would show a maximum, but one parallel to the plane y = 0 would 
show a minimum. A point such as S is known as a saddle point. The orientation 
of the ‘saddle’ in the xy-plane is irrelevant; it is as shown in the figure solely for 
ease of discussion. For any saddle point the function increases in some directions 
away from the point but decreases in other directions. 
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SSS 


Figure 5.2 Stationary points of a function of two variables. A minimum 
occurs at B, a maximum at P and a saddle point at S. 


For functions of two variables, such as the one shown, it should be clear that a 
necessary condition for a stationary point (maximum, minimum or saddle point) 
to occur is that 


of =0 and of =0. (5.21) 

Ox Oy 
The vanishing of the partial derivatives in directions parallel to the axes is enough 
to ensure that the partial derivative in any arbitrary direction is also zero. The 
latter can be considered as the superposition of two contributions, one along 
each axis; since both contributions are zero, so is the partial derivative in the 
arbitrary direction. This may be made more precise by considering the total 
differential 

ac : 
df = Fix + say. 
Using (5.21) we see that although the infinitesimal changes dx and dy can be 
chosen independently the change in the value of the infinitesimal function df is 
always zero at a stationary point. 
We now turn our attention to determining the nature of a stationary point of 

a function of two variables, i.e. whether it is a maximum, a minimum or a saddle 
point. By analogy with the one-variable case we see that 6°f/dx* and 67f /dy* 
must both be positive for a minimum and both be negative for a maximum. 
However these are not sufficient conditions since they could also be obeyed at 
complicated saddle points. What is important for a minimum (or maximum) is 
that the second partial derivative must be positive (or negative) in all directions, 
not just in the x- and y- directions. 
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To establish just what constitutes sufficient conditions we first note that, since 
f is a function of two variables and éf/éx = df /dy = 0, a Taylor expansion of 
the type (5.18) about the stationary point yields 


1 
f(x,y) — flx0, Yo) © 5 [(Ax)" fix + 2AxAy fey + (Ay) fyy] » 


where Ax = x — Xp and Ay = y — yo and where the partial derivatives have been 
written in more compact notation. Rearranging the contents of the bracket as 
the weighted sum of two squares, we find 


A 2 2 : 
f(x,y) flo, 90) © 5 [i (ax+ 2%) +cay? (5 - | 3 
XX xx (5.22) 


For a minimum, we require (5.22) to be positive for all Ax and Ay, and hence 
fxx > 0 and fyy — (£2, / fx) > 0. Given the first constraint, the second can be 
written fxxfyy > ee Similarly for a maximum we require (5.22) to be negative, 
and hence f,, <0 and fixfyy > {z,. For minima and maxima, symmetry requires 
that f}, obeys the same criteria as f,,. When (5.22) is negative (or zero) for some 
values of Ax and Ay but positive (or zero) for others, we have a saddle point. In 
this case fixfyy < f2,. In summary, all stationary points have f, = fy =0 and 
they may be classified further as 


(i) minima if both f,, and f,, are positive and cee < faxfyys 
(ii) maxima if both fy, and fy, are negative and fee < foxfyys 


(iii) saddle points if f,. and f,, have opposite signs or fj, > fixfyy. 


Note, however, that if ree = frxfyy then f(x,y) — f(x0, yo) can be written in one 
of the four forms 


1 ; a 
$5 (Axial? + Avlfiylt?) 


For some choice of the ratio Ay/Ax this expression has zero value, showing 
that, for a displacement from the stationary point in this particular direction, 
f (xo + Ax, yo + Ay) does not differ from f(xo, yo) to second order in Ax and 
Ay; in such situations further investigation is required. In particular, if f., fyy 
and f,, are all zero then the Taylor expansion has to be taken to a higher 
order. As examples, such extended investigations would show that the function 
f(x,y) = x4 + y* has a minimum at the origin but that g(x,y) = x++ y? has a 
saddle point there. 
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> Show that the function f(x, y) = x° exp(—x*— y”) has a maximum at the point (,/3/2,0), 
a minimum at (—1/3/2, 0) and a stationary point at the origin whose nature cannot be 
determined by the above procedures. 


Setting the first two partial derivatives to zero to locate the stationary points, we find 


of (3x? — 2x*) exp(—x? — y”) =0 (5.23) 
ox : 

a 2yx? exp(—x? — y”) = 0. (5.24) 
oy 


For (5.24) to be satisfied we require x = 0 or y = 0 and for (5.23) to be satisfied we require 
x=O0orx= +,/3/2. Hence the stationary points are at (0,0), (,/3/2,0) and (—\/3/2,0). 
We now find the second partial derivatives: 

fox = (4x° — 14x? + 6x) exp(—x’ — y”), 

fy = x°(4y’ — 2) exp(—x’ — y’), 

fay = 2x7y(2x? — 3) exp(—x? — y?), 


We then substitute the pairs of values of x and y for each stationary point and find that 
at (0,0) 


and at (+,/3/2,0) 


fee = F6y/3/2 exp(—3/2), fy = F3/3/2 exp(—3/2), fry = 0. 


Hence, applying criteria (i)-(iii) above, we find that (0,0) is an undetermined stationary 
point, (,/3/2,0) is a maximum and (—,/3/2,0) is a minimum. The function is shown in 
figure 5.3. < 


Determining the nature of stationary points for functions of a general number 
of variables is considerably more difficult and requires a knowledge of the 
eigenvectors and eigenvalues of matrices. Although these are not discussed until 
chapter 8, we present the analysis here for completeness. The remainder of this 
section can therefore be omitted on a first reading. 

For a function of n real variables, f(x1,X2,...,Xn), we require that, at all 
stationary points, 

of 


=—=0 for all x;. 
OX; 


In order to determine the nature of a stationary point, we must expand the 


function as a Taylor series about the point. Recalling the Taylor expansion (5.20) 
for a function of n variables, we see that 


Ae 1 ar 
Af = f(x) — f(x0) ® 5 Soo Ax AX. (5.25) 


— £4 0X;0X; 
ij 
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maximum 


Figure 5.3. The function f(x, y) = x? exp(—x? — y). 


If we define the matrix M to have elements given by 


Of 


ij 
i Oxi0x;" 


then we can rewrite (5.25) as 
Af = 4Ax™MAx, (5.26) 


where Ax is the column vector with the Ax; as its components and Ax! is its 
transpose. Since M is real and symmetric it has n real eigenvalues /, and n 
orthogonal eigenvectors e,, which after suitable normalisation satisfy 


T 
Me, = 4,e;, e, Cs = Ors, 


where the Kronecker delta, written 6,;, equals unity for r = s and equals zero 
otherwise. These eigenvectors form a basis set for the n-dimensional space and 
we can therefore expand Ax in terms of them, obtaining 


Ax = ) 4,2; 
; 
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where the a, are coefficients dependent upon Ax. Substituting this into (5.26), we 
find 
Af = 3Ax™MAx = 35° A,a7. 
r 

Now, for the stationary point to be a minimum, we require Af = $y 2,a2 >0 
for all sets of values of the a,, and therefore all the eigenvalues of M to be 
greater than zero. Conversely, for a maximum we require Af = ear a2 <0, 
and therefore all the eigenvalues of M to be less than zero. If the eigenvalues have 
mixed signs, then we have a saddle point. Note that the test may fail if some or 
all of the eigenvalues are equal to zero and all the non-zero ones have the same 
sign. 


> Derive the conditions for maxima, minima and saddle points for a function of two real 
variables, using the above analysis. 


For a two-variable function the matrix M is given by 


a fx fry 
fs ( fox fry : 


Therefore its eigenvalues satisfy the equation 


fo fry 
fry Fry —A 


Hence 


(fax — Afyy — A — fey =0 


> faxfyy ~ (fx + fry) ae 2 = Sy =0 


=> = (fet fy) t Cu + fol? — aby — 3) 


which by rearrangement of the terms under the square root gives 


2A= (fox + fry) + V (fox — fy)? + 4fi,. 


Now, that M is real and symmetric implies that its eigenvalues are real, and so for both 
eigenvalues to be positive (corresponding to a minimum), we require fy, and f,, positive 
and also 


fect fay > [Feet fy? —Mfehy — PR) 
aad frxfyy — s, > 0. 


A similar procedure will find the criteria for maxima and saddle points. < 


5.9 Stationary values under constraints 


In the previous section we looked at the problem of finding stationary values of 
a function of two or more variables when all the variables may be independently 
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varied. However, it is often the case in physical problems that not all the vari- 
ables used to describe a situation are in fact independent, i.e. some relationship 
between the variables must be satisfied. For example, if we walk through a hilly 
landscape and we are constrained to walk along a path, we will never reach 
the highest peak on the landscape unless the path happens to take us to it. 
Nevertheless, we can still find the highest point that we have reached during our 
journey. 

We first discuss the case of a function of just two variables. Let us consider 
finding the maximum value of the differentiable function f(x,y) subject to the 
constraint g(x,y) = c, where c is a constant. In the above analogy, f(x,y) might 
represent the height of the land above sea-level in some hilly region, whilst 
g(x, y) =c is the equation of the path along which we walk. 

We could, of course, use the constraint g(x,y) = c to substitute for x or y in 
f(x,y), thereby obtaining a new function of only one variable whose stationary 
points could be found using the methods discussed in subsection 2.1.8. However, 
such a procedure can involve a lot of algebra and becomes very tedious for func- 
tions of more than two variables. A more direct method for solving such problems 
is the method of Lagrange undetermined multipliers, which we now discuss. 

To maximise f we require 


of 


df = 3x ox + of 


By —dy =0. 


If dx and dy were independent, we could conclude f, = 0 = f,. However, here 
they are not independent, but constrained because g is constant: 


dg og 
dg = “2 dx 4+ Bay =0. 
cae ie Togs 


Multiplying dg by an as yet unknown number / and adding it to df we obtain 


d(f + 4g) = (< +138) a+ (5 +18) dy =0, 


where 4 is called a Lagrange undetermined multiplier. In this equation dx and dy 
are to be independent and arbitrary; we must therefore choose 4 such that 


; 
Of , 488 6 (5.27) 
Ox Ox 

of | 0g 

ay t 45 =o (5.28) 


These equations, together with the constraint g(x, y) = c, are sufficient to find the 
three unknowns, i.e. 2 and the values of x and y at the stationary point. 
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> The temperature of a point (x,y) on a unit circle is given by T(x, y) = 1+ xy. Find the 
temperature of the two hottest points on the circle. 


We need to maximise T(x, y) subject to the constraint x? + y? = 1. Applying (5.27) and 
(5.28), we obtain 


y+2ix =0, (5.29) 
x+21y =0. (5.30) 


These results, together with the original constraint x? + y? = 1, provide three simultaneous 
equations that may be solved for 4, x and y. 

From (5.29) and (5.30) we find 2 = +1/2, which in turn implies that y = +x. Remem- 
bering that x? + y? = 1, we find that 


We have not yet determined which of these stationary points are maxima and which are 
minima. In this simple case, we need only substitute the four pairs of x- and y- values into 
T(x, y) =1+xy to find that the maximum temperature on the unit circle is Tax = 3/2 at 


the points y=x = +1/,/2. < 


The method of Lagrange multipliers can be used to find the stationary points of 
functions of more than two variables, subject to several constraints, provided that 
the number of constraints is smaller than the number of variables. For example, 
if we wish to find the stationary points of f(x,y,z) subject to the constraints 
g(x, y,z) =c, and h(x, y,z) = cy, where c; and c) are constants, then we proceed 
as above, obtaining 


a. _ of . 0g, oh | 

ax +48 t+ uh) = 5 tAa tus = 8, 

a af .dg oh 

OG tig hays ee! ee pa BG 5.31 
ay g + uh) ay AG ey , (5.31) 
a. _ of dg, oh _ 

a eae ue) dz “dz | Maz a 


We may now solve these three equations, together with the two constraints, to 
give A, w, x, y and z. 
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> Find the stationary points of f(x,y,z) = x°+y>+z°> subject to the following constraints: 


(i) g(x y,z) =x? +y? +2? =1; 


(ii) g(x, y,z) =x? +y? +27 =1 and h(x,y,z)=x+ty+z=0. 


Case (i). Since there is only one constraint in this case, we need only introduce a single 
Lagrange multiplier to obtain 


: 
A(f +g) = 3x2 + 2Ax = 0, 
Ox 


au + Ag) = 3y? + 2ay =0, (5.32) 


6 2 
Syl + Ag) = 32? + 22z =0. 


These equations are highly symmetrical and clearly have the solution x = y = z = —22/3. 
Using the constraint x? + y? + z? = 1 we find 2 = +./3/2 and so stationary points occur 
at 


Yeyper= (5.33) 


7 
3 
In solving the three equations (5.32) in this way, however, we have implicitly assumed 

that x, y and z are non-zero. However, it is clear from (5.32) that any of these values can 

equal zero, with the exception of the case x = y = z = 0 since this is prohibited by the 
constraint x? + y? +z? = 1. We must consider the other cases separately. 
If x = 0, for example, we require 


3y? + 2dy = 0, 

327 + 24z = 0, 

yo2=1. 
Clearly, we require 2 # 0, otherwise these equations are inconsistent. If neither y nor 
z is zero we find y = —2A/3 = z and from the third equation we require y = z = 


+1/,/2. If y = 0, however, then z = +1 and, similarly, if z = 0 then y = +1. Thus the 
stationary points having x = 0 are (0,0,+1), (0,+1,0) and (0, +1//2, +1/,/2). A similar 
procedure can be followed for the cases y = 0 and z = 0 respectively and, in addition 
to those already obtained, we find the stationary points (+1,0,0), (4 1//2, 0, 4 1/,/2) and 
(t1//2, +1//2, 0). 

Case (ii). We now have two constraints and must therefore introduce two Lagrange 
multipliers to obtain (cf. (5.31)) 


at + Ag + wh) = 3x? + 24x + wp =0, (5.34) 
ai NSB iy eS. (5.35) 
a 

al 4+ dg + ph) = 32? +22+p=0. (5.36) 


These equations are again highly symmetrical and the simplest way to proceed is to 
subtract (5.35) from (5.34) to obtain 


3(x? — y?) + 2A(x— y) =0 
=> 3(x+yi(x—y)+2A(x—y) =0. (5.37) 


This equation is clearly satisfied if x = y; then, from the second constraint, x + y +z =0, 
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we find z = —2x. Substituting these values into the first constraint, x7 + y? + z? = 1, we 
obtain 


1 1 2 
att, yott, raed, (538) 


ig We ve 


Because of the high degree of symmetry amongst the equations (5.34)-(5.36), we may obtain 
by inspection two further relations analogous to (5.37), one containing the variables y,z 
and the other the variables x,z. Assuming y = z in the first relation and x = z in the 
second, we find the stationary points 


1 2 
t=, p= Fs, + 5.39 
reg ag * Ie rae 
and 
2 1 1 
po, +, +—. 5.40 
te Sagi egg aa ee 


We note that in finding the stationary points (5.38)-(5.40) we did not need to evaluate the 
Lagrange multipliers 2 and y explicitly. This is not always the case, however, and in some 
problems it may be simpler to begin by finding the values of these multipliers. 

Returning to (5.37) we must now consider the case where x # y; then we find 


3(x + y) +24 =0. (5.41) 


However, in obtaining the stationary points (5.39), (5.40), we did not assume x = y but 
only required y = z and x =z respectively. It is clear that x # y at these stationary points, 
and it can be shown that they do indeed satisfy (5.41). Similarly, several stationary points 
for which x #z or y #z have already been found. 

Thus we need to consider further only two cases, x = y = z, and x, y and z are all 
different. The first is clearly prohibited by the constraint x + y-+z = 0. For the second 
case, (5.41) must be satisfied, together with the analogous equations containing y,z and 
X,Z respectively, i.e. 


3(x + y) +24 =0, 
3(y +z) +22 =0, 
3(x +2) +22 =0. 


Adding these three equations together and using the constraint x+y+z = 0 we find 2 = 0. 
However, for 2 = 0 the equations are inconsistent for non-zero x, y and z. Therefore all 
the stationary points have already been found and are given by (5.38)-(5.40). < 


The method may be extended to functions of any number n of variables 
subject to any smaller number m of constraints. This means that effectively there 
are n—™m independent variables and, as mentioned above, we could solve by 
substitution and then by the methods of the previous section. However, for large 
n this becomes cumbersome and the use of Lagrange undetermined multipliers is 
a useful simplification. 
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>A system contains a very large number N of particles, each of which can be in any of R 
energy levels with a corresponding energy E;, i= 1,2,...,R. The number of particles in the 
ith level is n; and the total energy of the system is a constant, E. Find the distribution of 


particles amongst the energy levels that maximises the expression 
N! 
subject to the constraints that both the number of particles and the total energy remain 


constant, i.e. 


R 
g=N-)J 1, =0 and h=E-)_ nE,=0. 
i=1 ; 


The way in which we proceed is as follows. In order to maximise P, we must minimise 
its denominator (since the numerator is fixed). Minimising the denominator is the same as 
minimising the logarithm of the denominator, i.e. 


f =In(nj!ny!-+- ne!) = In(n!) + In(m!) +--+ + In (ng!). 
Using Stirling’s approximation, In (n!) ~ nlnn— n, we find that 


f =m Inn, +m inn +--+ +nglnng — (ny +n +--+ +7) 


R 
= (3m tan) —N. 
i=l 


It has been assumed here that, for the desired distribution, all the n; are large. Thus, we 
now have a function f subject to two constraints, g = 0 and h = 0, and we can apply the 
Lagrange method, obtaining (cf. (5.31)) 


of .ég oh 
Bi pacer 0 
on, on, Yon ; 
of .ég oh 
4+) 4 yp =0 
ony ony Yoh : 


A5— TH 
Onr Ong Ong 


Since all these equations are alike, we consider the general case 


0 0 Oh 
ul peers b— = 0, 
On Ong On 


for k = 1,2,...,R. Substituting the functions f, g and h into this relation we find 


7A + Inn + A(—-1) + n(—Ex) = 0, 


Nk 
which can be rearranged to give 
Inn = wE, +4-—1, 
and hence 


ny = Cexp wE,. 
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We now have the general form for the distribution of particles amongst energy levels, but 
in order to determine the two constants pu, C we recall that 


R 
So CexpyE =N 


k=1 


and 
R 


a CE, exp wE, = E. 


k=1 


This is known as the Boltzmann distribution and is a well-known result from statistical 
mechanics. <¢ 


5.10 Envelopes 


As noted at the start of this chapter, many of the functions with which physicists, 
chemists and engineers have to deal contain, in addition to constants and one 
or more variables, quantities that are normally considered as parameters of the 
system under study. Such parameters may, for example, represent the capacitance 
of a capacitor, the length of a rod, or the mass of a particle — quantities that 
are normally taken as fixed for any particular physical set-up. The corresponding 
variables may well be time, currents, charges, positions and velocities. However, 
the parameters could be varied and in this section we study the effects of doing so; 
in particular we study how the form of dependence of one variable on another, 
typically y = y(x), is affected when the value of a parameter is changed in a 
smooth and continuous way. In effect, we are making the parameter into an 
additional variable. 

As a particular parameter, which we denote by «, is varied over its permitted 
range, the shape of the plot of y against x will change, usually, but not always, 
in a smooth and continuous way. For example, if the muzzle speed v of a shell 
fired from a gun is increased through a range of values then its height—distance 
trajectories will be a series of curves with a common starting point that are 
essentially just magnified copies of the original; furthermore the curves do not 
cross each other. However, if the muzzle speed is kept constant but 0, the angle 
of elevation of the gun, is increased through a series of values, the corresponding 
trajectories do not vary in a monotonic way. When 0 has been increased beyond 
45° the trajectories then do cross some of the trajectories corresponding to 0 < 45°. 
The trajectories for 0 > 45° all lie within a curve that touches each individual 
trajectory at one point. Such a curve is called the envelope to the set of trajectory 
solutions; it is to the study of such envelopes that this section is devoted. 

For our general discussion of envelopes we will consider an equation of the 
form f = f(x, y, «) = 0. A function of three Cartesian variables, f = f(x, y, «), 
is defined at all points in xya-space, whereas f = f(x, y, ~) = 0 is a surface in 
this space. A plane of constant «, which is parallel to the xy-plane, cuts such 
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F%Y%)=0 feo +h) =0 
x 


Figure 5.4 Two neighbouring curves in the xy-plane of the family f(x,y, «) = 
0 intersecting at P. For fixed «1, the point P; is the limiting position of P as 
h — 0. As a is varied, P, delineates the envelope of the family (broken line). 


a surface in a curve. Thus different values of the parameter « correspond to 
different curves, which can be plotted in the xy-plane. We now investigate how 
the envelope equation for such a family of curves is obtained. 


5.10.1 Envelope equations 


Suppose f(x, y,%,) = 0 and f(x, y,a; +h) = 0 are two neighbouring curves of a 
family for which the parameter « differs by a small amount h. Let them intersect 
at the point P with coordinates x,y, as shown in figure 5.4. Then the envelope, 
indicated by the broken line in the figure, touches f(x, y,a,) =0 at the point P,, 
which is defined as the limiting position of P when «, is fixed but h — 0. The 
full envelope is the curve traced out by P; as a changes to generate successive 
members of the family of curves. Of course, for any finite h, f(x, y,a; +h) = 0 is 
one of these curves and the envelope touches it at the point P). 

We are now going to apply Rolle’s theorem, see subsection 2.1.10, with the 
parameter « as the independent variable and x and y fixed as constants. In this 
context, the two curves in figure 5.4 can be thought of as the projections onto the 
xy-plane of the planar curves in which the surface f = f(x, y, a) = 0 meets the 
planes a = a, anda =a, +h. 

Along the normal to the page that passes through P, as « changes from 
to a, +h the value of f = f(x, y, «) will depart from zero, because the normal 
meets the surface f = f(x, y, ~) = 0 only at « = a and at « = a +h. However, 
at these end points the values of f = f(x, y, «) will both be zero, and therefore 
equal. This allows us to apply Rolle’s theorem and so to conclude that for some 
0 in the range 0 < @ < 1 the partial derivative 0f(x, y, «, + 0h)/du is zero. When 
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h is made arbitrarily small, so that P > P;, the three defining equations reduce 
to two, which define the envelope point P;: 


f(xy,a1) =0 and Cie a) =a (5.42) 
In (5.42) both the function and the gradient are evaluated at « = a. The equation 
of the envelope g(x,y) = 0 is found by eliminating «; between the two equations. 
As a simple example we will now solve the problem which when posed mathe- 
matically reads ‘calculate the envelope appropriate to the family of straight lines 
in the xy-plane whose points of intersection with the coordinate axes are a fixed 
distance apart’. In more ordinary language, the problem is about a ladder leaning 
against a wall. 


>A ladder of length L stands on level ground and can be leaned at any angle against a 
vertical wall. Find the equation of the curve bounding the vertical area below the ladder. 


We take the ground and the wall as the x- and y-axes respectively. If the foot of the ladder 
is a from the foot of the wall and the top is b above the ground then the straight-line 
equation of the ladder is 


xy 

Ril ce eee 
a _ b ; 

where a and b are connected by a? + b? = L?. Expressed in standard form with only one 

independent parameter, a, the equation becomes 


y 


poe (5.43) 


x 
f(x y,4) a at 


Now, differentiating (5.43) with respect to a and setting the derivative 0f/da equal to 
zero gives 


from which it follows that 


Lx! 
= G23 + ya 


Ly'3 
x2/3 4 y2/3)1/2" 
(x73 + y?/3) 


and (L? a’)? = 
Eliminating a by substituting these values into (5.43) gives, for the equation of the 

envelope of all possible positions on the ladder, 

2/3 4 2/3 = 13/3, 


This is the equation of an astroid (mentioned in exercise 2.19), and, together with the wall 
and the ground, marks the boundary of the vertical area below the ladder. < 


Other examples, drawn from both geometry and and the physical sciences, are 
considered in the exercises at the end of this chapter. The shell trajectory problem 
discussed earlier in this section is solved there, but in the guise of a question 
about the water bell of an ornamental fountain. 
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5.11 Thermodynamic relations 


Thermodynamic relations provide a useful set of physical examples of partial 
differentiation. The relations we will derive are called Maxwell’s thermodynamic 
relations. They express relationships between four thermodynamic quantities de- 
scribing a unit mass of a substance. The quantities are the pressure P, the volume 
V, the thermodynamic temperature T and the entropy S of the substance. These 
four quantities are not independent; any two of them can be varied independently, 
but the other two are then determined. 
The first law of thermodynamics may be expressed as 


dU =T dS — Pav, (5.44) 


where U is the internal energy of the substance. Essentially this is a conservation 
of energy equation, but we shall concern ourselves, not with the physics, but rather 
with the use of partial differentials to relate the four basic quantities discussed 
above. The method involves writing a total differential, dU say, in terms of the 
differentials of two variables, say X and Y, thus 


0U 0U 
w= (22) ax (¥) a, 45 


and then using the relationship 
o°U o°U 
OXOY aYOX 
to obtain the required Maxwell relation. The variables X and Y are to be chosen 
from P, V, T and S. 


> Show that (0T/dV)s = —(0P/0S)y. 


Here the two variables that have to be held constant, in turn, happen to be those whose 
differentials appear on the RHS of (5.44). And so, taking X as S and Y as V in (5.45), we 


have 
TdS —PdV =dU= () s+ (3) dV, 
oS Jy OV) s 


A A 
(S) =T and (=) =—P. 
OS ]y OV) s 


Differentiating the first expression with respect to V and the second with respect to S, and 
using 


and find directly that 


we find the Maxwell relation 


5.11 THERMODYNAMIC RELATIONS 


> Show that (0S/éV)r = (6P/eT)y. 


Applying (5.45) to dS, with independent variables V and T, we find 


A } 
au =Tas—pav=T|(2)\ av+(2) ar| —pav. 
OV) ¢ oT /y, 


Similarly applying (5.45) to dU, we find 


0U o0U 
7 4 I 


Thus, equating partial derivatives, 
7) (+) (=) (=) 
=) =T|~=) -P and —) =Ti|=]} . 
(+ r OV) > oT), oT), 


fU  &U : é 
6TOV 8VaT’ 6T 


But, since 


it follows that 


Cre ore, 
Vjx @TeV  \aT)y eV 
Thus finally we get the Maxwell relation 
(= 2) 
Vis NOES. 
The above derivation is rather cumbersome, however, and a useful trick that 
can simplify the working is to define a new function, called a potential. The 


internal energy U discussed above is one example of a potential but three others 
are commonly defined and they are described below. 


> Show that (OS/0V)r; = (0P/0T)y by considering the potential U— ST. 


We first consider the differential d(U — ST). From (5.5), we obtain 
d(U — ST) =dU — SdT — TdS SdT — PdvV 


when use is made of (5.44). We rewrite U — ST as F for convenience of notation; F is 
called the Helmholtz potential. Thus 


dF =—SdT —Pdv, 


OF OF 
(=) Lo as (+), = 


Using these results together with 


and it follows that 


we can see immediately that 


(37) ,~ (sr), 


which is the same Maxwell relation as before. < 
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Although the Helmholtz potential has other uses, in this context it has simply 
provided a means for a quick derivation of the Maxwell relation. The other 
Maxwell relations can be derived similarly by using two other potentials, the 
enthalpy, H = U+ PV, and the Gibbs free energy, G = U+ PV — ST (see 
exercise 5.25). 


5.12 Differentiation of integrals 


We conclude this chapter with a discussion of the differentiation of integrals. Let 
us consider the indefinite integral (cf. equation (2.30)) 


F(x,t) = [f6. t) dt. 


from which it follows immediately that 


ons t) 


= f(x, t). 
Assuming that the second partial derivatives of F(x,t) are continuous, we have 


O°F(x,t) 0? F(x, t) 


otox Oxot 


and so we can write 


a [cr] _é Ee 9] _ of (x,t) 


ot Ox Ox ot Ox 
Integrating this equation with respect to t then gives 


OF (x, t) = (ea 1) i 


Ox Ox 


(5.46) 


Now consider the definite integral 


t=v 


I(x) = f(x, t) dt 


t=u 
_ F(x,v) oe F(x,u), 
where u and v are constants. Differentiating this integral with respect to x, and 
using (5.46), we see that 


dI(x)  OF(x,v) OF (x,u) 


dx Ox Ox 
aft), [Sf 1 
-[ Ox -| Ox 
fst) 


ae 


This is Leibnitz’ rule for differentiating integrals, and basically it states that for 
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constant limits of integration the order of integration and differentiation can be 
reversed. 

In the more general case where the limits of the integral are themselves functions 
of x, it follows immediately that 


= F(x,v(x)) — F(x, u(x)), 


which yields the partial derivatives 


= flso(y), = —Fosu(x). 
ou 


di (dal dv | ol Gk gs 
dx dv) dx \éu) dx | éx 


1 
= flx,vyZ ~ fooueoy A Ho flx.ndt 


Consequently 


; dv : du ) OF (x,t) 
= fesoty se —fesmceyg + [AEP ar (5.47) 
where the partial derivative with respect to x in the last term has been taken 
inside the integral sign using (5.46). This procedure is valid because u(x) and v(x) 
are being held constant in this term. 


> Find the derivative with respect to x of the integral 


2 
me io t 
I(x) = ‘) “ dt. 


Applying (5.47), we see that 


= a 


a sin x? (ox sin x (4) T tcos xt 
dx x x 7 t 


es 
2sinx? — sin x? = “1 
x 


x x x 
sin x sin x 
3 2. 
x x 


1 
= so sin x? — 2sin x’). < 


5.13 Exercises 


5.1 Using the appropriate properties of ordinary derivatives, perform the following. 
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52 


5.3 


5.4 


5.5 


5.6 


5.7 


5.8 


(a) Find all the first partial derivatives of the following functions f(x, y): 
(i) x’y, (ii) x? + y? +4, (iii) sin(x/y), (iv) tan7!(y/x), 
(v) rx, y,2) = OP + y? +27)?. 

(b) For (i), (ii) and (v), find 6?f/@x’, é?f /dy? and 67f /dxéy. 

(c) For (iv) verify that 6?f/dxdy = 67f /eyéx. 

Determine which of the following are exact differentials: 


a) (3x +2)ydx + x(x + 1) dy; 

b) ytanxdx+xtanydy; 

c) y?(Inx + 1)dx + 2xyInx dy; 

d) y?(Inx + 1) dy + 2xy Inx dx; 

e) [x/O? + yl] dy — [y/0? + y)] dx. 


Show that the differential 


( 
( 
( 
( 
( 


df =x? dy —(y? +xy)dx 
is not exact, but that dg = (xy”)"'df is exact. 
Show that 
df =y(1+x—x*)dx+x(x+4+ 1)dy 
is not an exact differential. 

Find the differential equation that a function g(x) must satisfy if dp = g(x)df 
is to be an exact differential. Verify that g(x) = e~* is a solution of this equation 
and deduce the form of (x, y). 

The equation 3y = z*+3xz defines z implicitly as a function of x and y. Evaluate 

all three second partial derivatives of z with respect to x and/or y. Verify that z 
is a solution of 

” ez &z 

Oy? 0x? 


A possible equation of state for a gas takes the form 


o 
PV =RTexp(-—), 

ee RE 
in which « and R are constants. Calculate expressions for 


oP av éT 
WV)» OT) 5 oP) ,’ 


and show that their product is —1, as stated in section 5.4. 
The function G(t) is defined by 


G(t) = F(x, y) = x° +. y? + 3xy, 


where x(t) = at? and y(t) = 2at. Use the chain rule to find the values of (x, y) at 
which G(t) has stationary values as a function of t. Do any of them correspond 
to the stationary points of F(x, y) as a function of x and y? 

In the xy-plane, new coordinates s and t are defined by 


s=3(xt+y), t=3(x-y). 
Transform the equation 


ed a6 


a Oy 
into the new coordinates and deduce that its general solution can be written 


$(x, y) = f(x + y) + g(x —y), 
where f(u) and g(v) are arbitrary functions of u and v, respectively. 
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5.9 


5.10 


5.11 


5.12 


5.13 


5.14 


5.15 


5.16 


5.17 


5.18 


5.19 


The function f(x, y) satisfies the differential equation 


of | of 
$5, = 0 


By changing to new variables u = x* — y* and v = 2xy, show that f is, in fact, a 
function of x? — y? only. 
If x = e“cos@ and y = e“sin@, show that 


Hp ob » (Of , Of 
ae t apr OP ty (a+ =) 
where f(x,y) = $(u,0). 

Find and evaluate the maxima, minima and saddle points of the function 


f(xy) = xy? + y? = 1). 


Show that 
f(xy) =x? — 12xy + 48x + by’, b#0, 


has two, one, or zero stationary points, according to whether |b| is less than, 
equal to, or greater than 3. 
Locate the stationary points of the function 


f(x,y) = (X° — 2y’)exp[-(? + y’)/a’], 
where a is a non-zero constant. 
Sketch the function along the x- and y-axes and hence identify the nature and 
values of the stationary points. 
Find the stationary points of the function 


f(xy) = e+ xy? —12x-— y 


and identify their natures. 
Find the stationary values of 


f(y) = 4x? + 4y? + x4 — 6x7? + y* 


and classify them as maxima, minima or saddle points. Make a rough sketch of 
the contours of f in the quarter plane x, y > 0. 
The temperature of a point (x,y,z) on the unit sphere is given by 


T (x,y,z) =1+xy+ yz. 


By using the method of Lagrange multipliers, find the temperature of the hottest 
point on the sphere. 
A rectangular parallelepiped has all eight vertices on the ellipsoid 


x? + 3y? +32? =1. 


Using the symmetry of the parallelepiped about each of the planes x = 0, 
y = 0, z = 0, write down the surface area of the parallelepiped in terms of 
the coordinates of the vertex that lies in the octant x,y,z > 0. Hence find the 
maximum value of the surface area of such a parallelepiped. 

Two horizontal corridors, 0 < x < a with y> 0, andO0 < y < b with x > 0, meet 
at right angles. Find the length L of the longest ladder (considered as a stick) 
that may be carried horizontally around the corner. 

A barn is to be constructed with a uniform cross-sectional area A throughout 
its length. The cross-section is to be a rectangle of wall height h (fixed) and 
width w, surmounted by an isosceles triangular roof that makes an angle 6 with 
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5.21 


5.22 


5:25 


the horizontal. The cost of construction is « per unit height of wall and f per 
unit (slope) length of roof. Show that, irrespective of the values of « and f, to 
minimise costs w should be chosen to satisfy the equation 


w* = 16A(A — wh), 


and @ made such that 2 tan 20 = w/h. 
Show that the envelope of all concentric ellipses that have their axes along the 
x- and y-coordinate axes, and that have the sum of their semi-axes equal to a 
constant L, is the same curve (an astroid) as that found in the worked example 
in section 5.10. 
Find the area of the region covered by points on the lines 

xy 

a + b = 1, 
where the sum of any line’s intercepts on the coordinate axes is fixed and equal 
toc. 
Prove that the envelope of the circles whose diameters are those chords of a 
given circle that pass through a fixed point on its circumference, is the cardioid 


r =a(1+cos6). 


Here a is the radius of the given circle and (r,@) are the polar coordinates of the 
envelope. Take as the system parameter the angle ¢ between a chord and the 
polar axis from which @ is measured. 

A water feature contains a spray head at water level at the centre of a round 
basin. The head is in the form of a small hemisphere perforated by many evenly 
distributed small holes, through which water spurts out at the same speed, vo, in 
all directions. 


(a) What is the shape of the ‘water bell’ so formed? 
(b) What must be the minimum diameter of the bowl if no water is to be lost? 


In order to make a focussing mirror that concentrates parallel axial rays to one 
spot (or conversely forms a parallel beam from a point source), a parabolic shape 
should be adopted. If a mirror that is part of a circular cylinder or sphere were 
used, the light would be spread out along a curve. This curve is known as a 
caustic and is the envelope of the rays reflected from the mirror. Denoting by 0 
the angle which a typical incident axial ray makes with the normal to the mirror 
at the place where it is reflected, the geometry of reflection (the angle of incidence 
equals the angle of reflection) is shown in figure 5.5. 
Show that a parametric specification of the caustic is 


x = Rcos0(4+sin’ 0), y = Rsin’ 0, 
where R is the radius of curvature of the mirror. The curve is, in fact, part of an 
epicycloid. 
By considering the differential 
dG=d(U+ PV —ST), 


where G is the Gibbs free energy, P the pressure, V the volume, S the entropy 
and T the temperature of a system, and given further that the internal energy U 
satisfies 


dU =TdS — Pav, 
derive a Maxwell relation connecting (6V/0T)p and (0S/0P)r. 
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5.26 


5.27 


5.28 


Figure 5.5 The reflecting mirror discussed in exercise 5.24. 


Functions P(V,T), U(V,T) and S(V, T) are related by 
TdS =dU+Pdadv, 


where the symbols have the same meaning as in the previous question. The 
pressure P is known from experiment to have the form 
Tt T 
P= > + Tv 
in appropriate units. If 
U =aVT*+8T, 
where «, f, are constants (or, at least, do not depend on T or V), deduce that « 


must have a specific value, but that 6 may have any value. Find the corresponding 
form of S. 


As in the previous two exercises on the thermodynamics of a simple gas, the 
quantity dS = T—'(dU + P dV) is an exact differential. Use this to prove that 


0U oP 
(ar},-7 (ar), -? 


In the van der Waals model of a gas, P obeys the equation 


RT a 
Pa ee 
V—b Vv? 


where R, a and b are constants. Further, in the limit V — oo, the form of U 
becomes U = cT, where c is another constant. Find the complete expression for 
U(V,T). 

The entropy S(H, T), the magnetisation M(H, T) and the internal energy U(H, T) 
of a magnetic salt placed in a magnetic field of strength H, at temperature T, 
are connected by the equation 


TdS = dU — HdM. 
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By considering d(U — TS — HM) prove that 
OoM\ _ (és 
6T), \@H), 


M(H, T) = Mo[1 — exp(—a«H/T)]. 


For a particular salt, 


Show that if, at a fixed temperature, the applied field is increased from zero to a 
strength such that the magnetization of the salt is 3M, then the salt’s entropy 
decreases by an amount 


Mo 
—(3 — 1n4). 
ag an) 


Using the results of section 5.12, evaluate the integral 


CO B-XY Gi 
re | ee” sin x ie 
0 


x 


Hence show that 


The integral 


2 2 
i e** dx 
—oO 


has the value (2/«)'/?. Use this result to evaluate 
2 = 
J(n) -| xe™ dx, 
oo 
where n is a positive integer. Express your answer in terms of factorials. 


The function f(x) is differentiable and f(0) = 0. A second function g(y) is defined 
by 


,_ [” fx)dx 
gly) = if Wak 
Prove that 


a. [ df dx 
dy Jo dx /y—x 


For the case f(x) = x", prove that 


d'g 
a = 2(n!),/y. 
The functions f(x, t) and F(x) are defined by 
fxoj=e™, 


F(x) = i ” fle,t) dt. 


Verify, by explicit calculation, that 


dF * Of (x,t) 
ae =so.+ [ Aa dt. 
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5.33 


5.34 


5.35 


5.1 


If 
1(2) =i iter oe B 
0 


Inx 


what is the value of I(0)? Show that 


—x* = x*Inx, 


da 

and deduce that 
d 1 
Fo! (*) ary reg 


Hence prove that J(«) = In(1 + «). 
Find the derivative, with respect to x, of the integral 


3x 
1) = f exp xt dt. 


The function G(t,€) is defined for0<t< a by 


G(t,é) = —costsiné for Eé <t, 
re’) = sint cos é for € >t. 


Show that the function x(t) defined by 


x(t) = [ "Get (ae 


0 


satisfies the equation 


where f(t) can be any arbitrary (continuous) function. Show further that x(0) = 
[dx/dt],-, = 0, again for any f(t), but that the value of x(z) does depend upon 


the form of f(t). 


[The function G(t,é) is an example of a Green’s function, an important 
concept in the solution of differential equations and one studied extensively in 


later chapters. ] 


5.14 Hints and answers 


(a) (i) 2xy, x; (ii) 2x, 2y; (iii) y~! cos(x/y), (—x/y") cos(x/y); 


(iv) —y/(x? + y?),x/(x? + y7); (v) x/r, y/t, 2/1. 


(b) (i) 2y,0,2x; (ii) 2,2,05 (v) (7 +27) 3, 0? 4.27), — xy. 
(c) Both second derivatives are equal to (y? — x?)(x? + y?)~. 
2x # —2y — x. For g, both sides of equation (5.9) equal y~?. 


6°2/0x? = 2xz(z? +x), 6?2/dxéy = (2? —x)(2? +x), 072/dy? = —22(2z? +x). 


(0,0), (a/4,—a) and (16a, —8a). Only the saddle point at (0,0). 


The transformed equation is 2(x? + y*)éf /dv = 0; hence f does not depend on v. 
Maxima, equal to 1/8, at +(1/2,—1/2), minima, equal to —1/8, at +(1/2,1/2), 


saddle points, equalling 0, at (0,0), (0,1), (£1, 0). 
Maxima equal to a’e! 


point equalling 0 at (0,0). 


at (ta,0), minima equal to —2a’e~! at (0,+a), saddle 


Minimum at (0,0); saddle points at (+1,+1). To help with sketching the contours, 


determine the behaviour of g(x) = f(x, x). 


The Lagrange multiplier method gives z = y = x/2, for a maximal area of 4. 
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5.33 
5.35 


The cost always includes 2ah, which can therefore be ignored in the optimisation. 

With Lagrange multiplier 2, sin@ = Aw/(4f) and f sec 0 — 5Aw tan @ = Ah, leading 

to the stated results. 

The envelope of the lines x/a+y/(c—a)—1 = 0, as a is varied, is xt Jy = Je. 

Area = c’/6. 

(a) Using « = cot 0, where @ is the initial angle a jet makes with the vertical, the 
equation is f(z, p,%) = z—pa+[gp?(1+47)/(2v4)], and setting df /0a = 0 gives 
a = va/(gp). The water bell has a parabolic profile z = v3 /(2g) — gp*/(2v9). 

(b) Setting z = 0 gives the minimum diameter as 203 /g. 

Show that (6G/éP)r = V and (6G/dT)p = —S. From each result, obtain an 

expression for 07G/@T@P and equate these, giving (OV /0T)p = —(0S/0P)r 

Find expressions for (0S/0V)r and (0S/0T)y, and equate 0?S/0V0T with 

&S/dTOV. U(V,T) =cT —aV—. 

dI /dy = —Im[ ie exp(—xy + ix) dx] = —1/(1+ y?). Integrate dI /dy from 0 to oo. 

I(oo) = 0 and I(0) = J. 

Integrate the RHS of the equation by parts, before differentiating with respect 

to y. Repeated application of the method establishes the result for all orders of 

derivative. 

I(0) = 0; use Leibnitz’ rule. 

Write x(t) = —cost fi siné f(é)dé — sint f." cos é f(€) dé and differentiate each 

term as a product to obtain dx/dt. Obtain d?x/dt? in a similar way. Note that 

integrals that have equal lower and upper limits have value zero. The value of 


n) is ff sin € f(€) dé. 
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Multiple integrals 


For functions of several variables, just as we may consider derivatives with respect 
to two or more of them, so may the integral of the function with respect to more 
than one variable be formed. The formal definitions of such multiple integrals are 
extensions of that for a single variable, discussed in chapter 2. We first discuss 
double and triple integrals and illustrate some of their applications. We then 
consider changing the variables in multiple integrals and discuss some general 
properties of Jacobians. 


6.1 Double integrals 


For an integral involving two variables — a double integral — we have a function, 
f(x,y) say, to be integrated with respect to x and y between certain limits. These 
limits can usually be represented by a closed curve C bounding a region R in the 
xy-plane. Following the discussion of single integrals given in chapter 2, let us 
divide the region R into N subregions AR, of area AA,, p = 1,2,...,N, and let 
(Xp, p) be any point in subregion AR,. Now consider the sum 


N 
S=S° f(xp, yp) AAp, 


p=1 


and let N — oo as each of the areas AA, — 0. If the sum S tends to a unique 
limit, J, then this is called the double integral of f(x,y) over the region R and is 
written 


I= | f(x,y) dA, (6.1) 


where dA stands for the element of area in the xy-plane. By choosing the 
subregions to be small rectangles each of area AA = AxAy, and letting both Ax 
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Figure 6.1 A simple curve C in the xy-plane, enclosing a region R. 


and Ay — 0, we can also write the integral as 


I = [[ foanasdy, (6.2) 


where we have written out the element of area explicitly as the product of the 
two coordinate differentials (see figure 6.1). 

Some authors use a single integration symbol whatever the dimension of the 
integral; others use as many symbols as the dimension. In different circumstances 
both have their advantages. We will adopt the convention used in (6.1) and (6.2), 
that as many integration symbols will be used as differentials explicitly written. 

The form (6.2) gives us a clue as to how we may proceed in the evaluation 
of a double integral. Referring to figure 6.1, the limits on the integration may 
be written as an equation c(x, y) = 0 giving the boundary curve C. However, an 
explicit statement of the limits can be written in two distinct ways. 

One way of evaluating the integral is first to sum up the contributions from 
the small rectangular elemental areas in a horizontal strip of width dy (as shown 
in the figure) and then to combine the contributions of these horizontal strips to 
cover the region R. In this case, we write 


y=d X=Xx2(y) 
fe il { / floyd} dy, (63) 
yee xa) 


where x = x;(y) and x = x2(y) are the equations of the curves TSV and TUV 
respectively. This expression indicates that first f(x,y) is to be integrated with 
respect to x (treating y as a constant) between the values x = x;(y) and x = x2(y) 
and then the result, considered as a function of y, is to be integrated between the 
limits y = c and y = d. Thus the double integral is evaluated by expressing it in 
terms of two single integrals called iterated (or repeated) integrals. 
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An alternative way of evaluating the integral, however, is first to sum up the 
contributions from the elemental rectangles arranged into vertical strips and then 
to combine these vertical strips to cover the region R. We then write 


x=b y=ya(x) 
I -| {| fox»)dv} dx, (6.4) 
x=a y=yi(x) 


where y = y;(x) and y = y2(x) are the equations of the curves STU and SVU 
respectively. In going to (6.4) from (6.3), we have essentially interchanged the 
order of integration. 

In the discussion above we assumed that the curve C was such that any line 
parallel to either the x- or y-axis intersected C at most twice. In general, provided 
f(x,y) is continuous everywhere in R and the boundary curve C has this simple 
shape, the same result is obtained irrespective of the order of integration. In cases 
where the region R has a more complicated shape, it can usually be subdivided 
into smaller simpler regions R;, Ry etc. that satisfy this criterion. The double 
integral over R is then merely the sum of the double integrals over the subregions. 


> Evaluate the double integral 


I =i xy dx dy, 
R 


where R is the triangular area bounded by the lines x = 0, y =0 and x + y = 1. Reverse 
the order of integration and demonstrate that the same result is obtained. 


The area of integration is shown in figure 6.2. Suppose we choose to carry out the 
integration with respect to y first. With x fixed, the range of y is 0 to 1—x. We can 


therefore write 
x= y=1-x 
I -| if yay} dx 
x=0 y=0 


x=l 7 52,27 y=l-x 1 \2 2 
: 1- 1 
a, Ea dx [= py : 
x=0 2 Jy-0 0 2 60 
Alternatively, we may choose to perform the integration with respect to x first. With y 
fixed, the range of x is 0 to 1 — y, so we have 


y=l x=l-y 
I -| if vyaeh dy 
y=0 x=0 


y=l 3,,7 x=l-y 1 _ By 
-{ Ea os if “ _ dy a 
y=0 x=0 0 


As expected, we obtain the same result irrespective of the order of integration. < 


We may avoid the use of braces in expressions such as (6.3) and (6.4) by writing 


(6.4), for example, as 
b y2(x) 
t= fax fase, 
a yi(x) 


where it is understood that each integral symbol acts on everything to its right, 
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y 
1 
dy 
x+ty=1 
R 
0 
0 dx 1 x 


Figure 6.2 The triangular region whose sides are the axes x = 0, y = 0 and 
the line x+y =1. 


and that the order of integration is from right to left. So, in this example, the 
integrand f(x, y) is first to be integrated with respect to y and then with respect 
to x. With the double integral expressed in this way, we will no longer write the 
independent variables explicitly in the limits of integration, since the differential 
of the variable with respect to which we are integrating is always adjacent to the 
relevant integral sign. 

Using the order of integration in (6.3), we could also write the double integral as 


d x2(y) 
ps | dy / dx f(x,y). 
€ xi(y) 


Occasionally, however, interchange of the order of integration in a double integral 
is not permissible, as it yields a different result. For example, difficulties might 
arise if the region R were unbounded with some of the limits infinite, though in 
many cases involving infinite limits the same result is obtained whichever order 
of integration is used. Difficulties can also occur if the integrand f(x, y) has any 
discontinuities in the region R or on its boundary C. 


6.2 Triple integrals 


The above discussion for double integrals can easily be extended to triple integrals. 
Consider the function f(x, y,z) defined in a closed three-dimensional region R. 
Proceeding as we did for double integrals, let us divide the region R into N 
subregions AR, of volume AV,, p = 1,2,...,N, and let (xp, yp,2Zp) be any point in 
the subregion AR,. Now we form the sum 


N 
S= S> fp. yp Zp) AV, > 
p=1 
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and let N — oo as each of the volumes AV, — 0. If the sum S tends to a unique 
limit, I, then this is called the triple integral of f(x, y,z) over the region R and is 
written 


I= i foxy,z) dV, (6.5) 


where dV stands for the element of volume. By choosing the subregions to be 
small cuboids, each of volume AV = AxAyAz, and proceeding to the limit, we 
can also write the integral as 


T= [[[ 16 y,z) dx dy dz, (6.6) 


where we have written out the element of volume explicitly as the product of the 
three coordinate differentials. Extending the notation used for double integrals, 
we may write triple integrals as three iterated integrals, for example, 


x2 y2(x) 22(x,y) 
I -| ax [ ay | dz f(x, y,Z), 
x1 yi(x) 21 (x,y) 


where the limits on each of the integrals describe the values that x, y and z take 
on the boundary of the region R. As for double integrals, in most cases the order 
of integration does not affect the value of the integral. 

We can extend these ideas to define multiple integrals of higher dimensionality 
in a similar way. 


6.3 Applications of multiple integrals 


Multiple integrals have many uses in the physical sciences, since there are numer- 
ous physical quantities which can be written in terms of them. We now discuss a 
few of the more common examples. 


6.3.1 Areas and volumes 


Multiple integrals are often used in finding areas and volumes. For example, the 


integral 
sa farm ff ace 
R R 


is simply equal to the area of the region R. Similarly, if we consider the surface 
z = f(x,y) in three-dimensional Cartesian coordinates then the volume under this 
surface that stands vertically above the region R is given by the integral 


v= feda= ff seosavay 


where volumes above the xy-plane are counted as positive, and those below as 
negative. 
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Figure 6.3. The tetrahedron bounded by the coordinate surfaces and the 
plane x/a+ y/b + z/c = 1 is divided up into vertical slabs, the slabs into 
columns and the columns into small boxes. 


> Find the volume of the tetrahedron bounded by the three coordinate surfaces x = 0, y =0 
and z =0 and the plane x/a+y/b+2z/c=1. 


Referring to figure 6.3, the elemental volume of the shaded region is given by dV = z dx dy, 
and we must integrate over the triangular region R in the xy-plane whose sides are x = 0, 
y =O and y = b— bx/a. The total volume of the tetrahedron is therefore given by 


b—bx/a 
r= [fpwo= [af os--9 


y=b—bx/a 


Alternatively, we can write the volume of a three-dimensional region R as 


-fav- ff dx dy dz, (6.7) 
R R 


where the only difficulty occurs in setting the correct limits on each of the 
integrals. For the above example, writing the volume in this way corresponds to 
dividing the tetrahedron into elemental boxes of volume dx dy dz (as shown in 
figure 6.3); integration over z then adds up the boxes to form the shaded column 
in the figure. The limits of integration are z = 0 to z =c (1 y/b x/a), and 
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the total volume of the tetrahedron is given by 


a b—bx/a c(1—y/b—x/a) 
V = ax [ av [ dz, (6.8) 
0 0 0 


which clearly gives the same result as above. This method is illustrated further in 
the following example. 


> Find the volume of the region bounded by the paraboloid z = x* + y* and the plane 
2B — Dy, 


The required region is shown in figure 6.4. In order to write the volume of the region in 
the form (6.7), we must deduce the limits on each of the integrals. Since the integrations 
can be performed in any order, let us first divide the region into vertical slabs of thickness 
dy perpendicular to the y-axis, and then as shown in the figure we cut each slab into 
horizontal strips of height dz, and each strip into elemental boxes of volume dV = dx dy dz. 
Integrating first with respect to x (adding up the elemental boxes to get a horizontal strip), 
the limits on x are x Jz y? to x = \/z—y?. Now integrating with respect to z 
(adding up the strips to form a vertical slab) the limits on z are z = y? to z = 2y. Finally, 
integrating with respect to y (adding up the slabs to obtain the required region), the limits 
on y are y= 0 and y = 2, the solutions of the simultaneous equations z = 0? + y? and 
z = 2y. So the volume of the region is 


2 2y /z—y? 2 2y 
v=faf az [ dx= fay f dz 2./z— y? 
0 y? - fy 0 y2 
2 2 
= f ay [ey PS = ff ay $oy— yp”. 
0 ? 0 


The integral over y may be evaluated straightforwardly by making the substitution y = 
1+sinu, and gives V = 2/2. 4 


In general, when calculating the volume (area) of a region, the volume (area) 
elements need not be small boxes as in the previous example, but may be of any 
convenient shape. The latter is usually chosen to make evaluation of the integral 
as simple as possible. 


6.3.2. Masses, centres of mass and centroids 


It is sometimes necessary to calculate the mass of a given object having a non- 
uniform density. Symbolically, this mass is given simply by 


m= fam, 


where dM is the element of mass and the integral is taken over the extent of the 
object. For a solid three-dimensional body the element of mass is just dM = pdV, 
where dV is an element of volume and op is the variable density. For a laminar 
body (i.e. a uniform sheet of material) the element of mass is dM = o dA, where 
o is the mass per unit area of the body and dA is an area element. Finally, for 
a body in the form of a thin wire we have dM = Ads, where A is the mass per 
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dV =dxdydz 


x 


Figure 6.4 The region bounded by the paraboloid z = x? + y’ and the plane 
z = 2y is divided into vertical slabs, the slabs into horizontal strips and the 
strips into boxes. 


unit length and ds is an element of arc length along the wire. When evaluating 
the required integral, we are free to divide up the body into mass elements in 
the most convenient way, provided that over each mass element the density is 
approximately constant. 


> Find the mass of the tetrahedron bounded by the three coordinate surfaces and the plane 
x/a+y/b+z/c =1, if its density is given by p(x, y,z) = po(1 + x/a). 


From (6.8), we can immediately write down the mass of the tetrahedron as 


a b—bx/a c(1-y/b—x/a) 
x x 
M= f (1+) av= [aveo(1+2) f av [ dz, 


where we have taken the density outside the integrations with respect to z and y since it 
depends only on x. Therefore the integrations with respect to z and y proceed exactly as 
they did when finding the volume of the tetrahedron, and we have 


¢ x\ (bx? bx b 
M=em [ dx (1+=) (3-=+3). (6.9) 


We could have arrived at (6.9) more directly by dividing the tetrahedron into triangular 
slabs of thickness dx perpendicular to the x-axis (see figure 6.3), each of which is of 
constant density, since p depends on x alone. A slab at a position x has volume dV = 
ze(1 — x/a)(b — bx/a)dx and mass dM = pdV = po(1 + x/a)dV. Integrating over x we 
again obtain (6.9). This integral is easily evaluated and gives M = sabcpo. < 
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The coordinates of the centre of mass of a solid or laminar body may also be 
written as multiple integrals. The centre of mass of a body has coordinates x, y, 
Z given by the three equations 


where again dM is an element of mass as described above, x, y, z are the 
coordinates of the centre of mass of the element dM and the integrals are taken 
over the entire body. Obviously, for any body that lies entirely in, or is symmetrical 
about, the xy-plane (say), we immediately have 7 = 0. For completeness, we note 
that the three equations above can be written as the single vector equation (see 


chapter 7) 
1 
r= xy fram. 
M 


where r is the position vector of the body’s centre of mass with respect to the 
origin, r is the position vector of the centre of mass of the element dM and 
M = f dM is the total mass of the body. As previously, we may divide the body 
into the most convenient mass elements for evaluating the necessary integrals, 
provided each mass element is of constant density. 

We further note that the coordinates of the centroid of a body are defined as 
those of its centre of mass if the body had uniform density. 


> Find the centre of mass of the solid hemisphere bounded by the surfaces x? + y? +2? = a? 
and the xy-plane, assuming that it has a uniform density p. 


Referring to figure 6.5, we know from symmetry that the centre of mass must lie on 
the z-axis. Let us divide the hemisphere into volume elements that are circular slabs of 
thickness dz parallel to the xy-plane. For a slab at a height z, the mass of the element is 
dM = pdV = pn(a’? —z’) dz. Integrating over z, we find that the z-coordinate of the centre 
of mass of the hemisphere is given by 


z{ pn(a? — 27) dz Ss zpn(a’ — 2”) dz. 
0 0 


The integrals are easily evaluated and give Z = 3a/8. Since the hemisphere is of uniform 
density, this is also the position of its centroid. < 


6.3.3 Pappus’ theorems 


The theorems of Pappus (which are about seventeen centuries old) relate centroids 
to volumes of revolution and areas of surfaces, discussed in chapter 2, and may be 
useful for finding one quantity given another that can be calculated more easily. 
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Figure 6.5 The solid hemisphere bounded by the surfaces x? + y? +2? =a 
and the xy-plane. 


y 


x 


Figure 6.6 An area A in the xy-plane, which may be rotated about the x-axis 
to form a volume of revolution. 


If a plane area is rotated about an axis that does not intersect it then the solid 
so generated is called a volume of revolution. Pappus’ first theorem states that the 
volume of such a solid is given by the plane area A multiplied by the distance 
moved by its centroid (see figure 6.6). This may be proved by considering the 
definition of the centroid of the plane area as the position of the centre of mass 
if the density is uniform, so that 

j= 2 / ydA., 


Now the volume generated by rotating the plane area about the x-axis is given by 
V= [ox dA = 2nyA, 
which is the area multiplied by the distance moved by the centroid. 
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x 


Figure 6.7 A curve in the xy-plane, which may be rotated about the x-axis 
to form a surface of revolution. 


Pappus’ second theorem states that if a plane curve is rotated about a coplanar 
axis that does not intersect it then the area of the surface of revolution so generated 
is given by the length of the curve L multiplied by the distance moved by its 
centroid (see figure 6.7). This may be proved in a similar manner to the first 
theorem by considering the definition of the centroid of a plane curve, 


1 


and noting that the surface area generated is given by 


S= [orvas = 2npL, 


which is equal to the length of the curve multiplied by the distance moved by its 
centroid. 


> A semicircular uniform lamina is freely suspended from one of its corners. Show that its 
straight edge makes an angle of 23.0° with the vertical. 


Referring to figure 6.8, the suspended lamina will have its centre of gravity C vertically 
below the suspension point and its straight edge will make an angle 0 = tan7'(d/a) with 
the vertical, where 2a is the diameter of the semicircle and d is the distance of its centre 
of mass from the diameter. 

Since rotating the lamina about the diameter generates a sphere of volume tna, Pappus’ 
first theorem requires that 


tna? = 2nd x 37a. 


Hence d = # and 0 = tan”! + = 23.0°. < 
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Figure 6.8 Suspending a semicircular lamina from one of its corners. 


6.3.4 Moments of inertia 


For problems in rotational mechanics it is often necessary to calculate the moment 
of inertia of a body about a given axis. This is defined by the multiple integral 


r= [ram, 


where ! is the distance of a mass element dM from the axis. We may again choose 
mass elements convenient for evaluating the integral. In this case, however, in 
addition to elements of constant density we require all parts of each element to 
be at approximately the same distance from the axis about which the moment of 
inertia is required. 


> Find the moment of inertia of a uniform rectangular lamina of mass M with sides a and 
b about one of the sides of length b. 


Referring to figure 6.9, we wish to calculate the moment of inertia about the y-axis. 
We therefore divide the rectangular lamina into elemental strips parallel to the y-axis of 
width dx. The mass of such a strip is dM = obdx, where o is the mass per unit area of 
the lamina. The moment of inertia of a strip at a distance x from the y-axis is simply 
dl = x?dM = obx* dx. The total moment of inertia of the lamina about the y-axis is 


therefore 
a b 3. 
I -| abx? dx = oe 
0 3 


Since the total mass of the lamina is M = cab, we can write I = +Ma’. < 
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dM = oabdx 


dx a x 


Figure 6.9 A uniform rectangular lamina of mass M with sides a and b can 
be divided into vertical strips. 


6.3.5 Mean values of functions 


In chapter 2 we discussed average values for functions of a single variable. This 
is easily extended to functions of several variables. Let us consider, for example, 
a function f(x,y) defined in some region R of the xy-plane. Then the average 
value f of the function is given by 


j i dA = | f(x,y) dA. (6.10) 


This definition is easily extended to three (and higher) dimensions; if a function 
f(x, y,z) 1s defined in some three-dimensional region of space R then the average 
value f of the function is given by 


7 fav =f pooy.zyar. (6.11) 


>A tetrahedron is bounded by the three coordinate surfaces and the plane x/a+y/b+z/c = 
1 and has density p(x, y,z) = po(1 + x/a). Find the average value of the density. 


From (6.11), the average value of the density is given by 


p fav =f oxvzav. 
R R 


Now the integral on the LHS is just the volume of the tetrahedron, which we found in 
subsection 6.3.1 to be V = gabe, and the integral on the RHS is its mass M = sabcpo, 


calculated in subsection 6.3.2. Therefore p = M/V = 3 po. < 


6.4 Change of variables in multiple integrals 


It often happens that, either because of the form of the integrand involved or 
because of the boundary shape of the region of integration, it is desirable to 
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u = constant 


— v=constant 


Figure 6.10 A region of integration R overlaid with a grid formed by the 
family of curves u = constant and v = constant. The parallelogram KLMN 
defines the area element dA,,. 


express a multiple integral in terms of a new set of variables. We now consider 
how to do this. 


6.4.1 Change of variables in double integrals 


Let us begin by examining the change of variables in a double integral. Suppose 
that we require to change an integral 


r= f i: f(x,y) dx dy, 


in terms of coordinates x and y, into one expressed in new coordinates u and v, 
given in terms of x and y by differentiable equations u = u(x, y) and v = v(x, y) 
with inverses x = x(u,v) and y = y(u,v). The region R in the xy-plane and the 
curve C that bounds it will become a new region R’ and a new boundary C’ in 
the uv-plane, and so we must change the limits of integration accordingly. Also, 
the function f(x, y) becomes a new function g(u,v) of the new coordinates. 

Now the part of the integral that requires most consideration is the area element. 
In the xy-plane the element is the rectangular area dA,, = dxdy generated by 
constructing a grid of straight lines parallel to the x- and y- axes respectively. 
Our task is to determine the corresponding area element in the wv-coordinates. In 
general the corresponding element dA,, will not be the same shape as dA,,, but 
this does not matter since all elements are infinitesimally small and the value of 
the integrand is considered constant over them. Since the sides of the area element 
are infinitesimal, dA,,, will in general have the shape of a parallelogram. We can 
find the connection between dA,, and dA,, by considering the grid formed by the 
family of curves u = constant and v = constant, as shown in figure 6.10. Since v 
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is constant along the line element K L, the latter has components (6x/éu) du and 
(Oy/du) du in the directions of the x- and y-axes respectively. Similarly, since u 
is constant along the line element KN, the latter has corresponding components 
(0x/dv) dv and (éy/év) dv. Using the result for the area of a parallelogram given 
in chapter 7, we find that the area of the parallelogram K LMN is given by 

Ox , oy Ox , Oy 


Bu duq dv — on doz du 


Oxdy Oxdy 
du dv Ov Ou 


dAw = 


Defining the Jacobian of x, y with respect to u, v as 


O(x,y) _ Ox dy Ox dy 
O(u,v) Oudv dv dw 


we have 


The reader acquainted with determinants will notice that the Jacobian can also 
be written as the 2 x 2 determinant 


Ox oy 
_ Oxy) | du du 
~ O(uv) | ax ay 
dv. ov 


Such determinants can be evaluated using the methods of chapter 8. 
So, in summary, the relationship between the size of the area element generated 
by dx, dy and the size of the corresponding area element generated by du, dv is 


0 


0 


dx dy = 


(x,y) 
(iv) | dudv. 


This equality should be taken as meaning that when transforming from coordi- 
nates x,y to coordinates u,v, the area element dxdy should be replaced by the 
expression on the RHS of the above equality. Of course, the Jacobian can, and 
in general will, vary over the region of integration. We may express the double 
integral in either coordinate system as 


t= ff fosnasay= ff etue) 


When evaluating the integral in the new coordinate system, it is usually advisable 
to sketch the region of integration R’ in the wv-plane. 


0 


0 


(%y 1 du do. (6.12) 
(u,v) 
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> Evaluate the double integral 


1= ff (a+ VP +9) dx dy, 


where R is the region bounded by the circle x? + y? = a’. 


In Cartesian coordinates, the integral may be written 


a Jax 
t=fa fw (a+ Vx? +¥"), 


and can be calculated directly. However, because of the circular boundary of the integration 
region, a change of variables to plane polar coordinates p, ¢ is indicated. The relationship 
between Cartesian and plane polar coordinates is given by x = pcos¢ and y = psing. 
Using (6.12) we can therefore write 

A(x, y) 


t= [ftoro) a(p. 0) 


where R’ is the rectangular region in the pd-plane whose sides are p = 0, p =a, 6 = 0 
and ¢ = 22. The Jacobian is easily calculated, and we obtain 


dp d¢, 


_ O(x,y) | cosh sin b 2 2 
~ Op.8) | —psing poosp |~ Pros e+ sin #) =p 
So the relationship between the area elements in Cartesian and in plane polar coordinates is 


dx dy = pdpd@. 


Therefore, when expressed in plane polar coordinates, the integral is given by 


I = | [ (a+ prpdoas 


6.4.2 Evaluation of the integral I = fies e-™ dx 


ive 


By making a judicious change of variables, it is sometimes possible to evaluate 
an integral that would be intractable otherwise. An important example of this 
method is provided by the evaluation of the integral 


I= / e™ dx. 


Its value may be found by first constructing I?, as follows: 


P= | dx i ee? dy = ji : ax [ : dy ety") 
—00 —o0 —00 —00 
= II ety") dx dy, 
R 
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y 


Figure 6.11 The regions used to illustrate the convergence properties of the 
integral I(a) = [",e* dx asa o. 


where the region R is the whole xy-plane. Then, transforming to plane polar 
coordinates, we find 


2n roe) & 
P= /I e ? pdpdd = ap | dp pe” =2n [-+e7"] =T. 
; 5 0 0 


Therefore the original integral is given by I = ,/z. Because the integrand is an 
even function of x, it follows that the value of the integral from 0 to oo is simply 
Jt/2. 

We note, however, that unlike in all the previous examples, the regions of 
integration R and R’ are both infinite in extent (ie. unbounded). It is therefore 
prudent to derive this result more rigorously; this we do by considering the 


integral 
a 2 
I(a) -| e™ dx. 
=a 


P(a)= I e 47) dx dy, 
R 


where R is the square of side 2a centred on the origin. Referring to figure 6.11, 
since the integrand is always positive the value of the integral taken over the 
square lies between the value of the integral taken over the region bounded by 
the inner circle of radius a and the value of the integral taken over the outer 
circle of radius ./2a. Transforming to plane polar coordinates as above, we may 


We then have 
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R | 
q 
oe 
v=C) 
u = Ci Cc 
\ H S Q 
wa P W=C3 
= Soe We Ae 


x 


Figure 6.12 A three-dimensional region of integration R, showing an el- 
ement of volume in u,v,w coordinates formed by the coordinate surfaces 
u = constant, v = constant, w = constant. 


evaluate the integrals over the inner and outer circles respectively, and we find 
T (1 _ e*) <I?(a)<1 (1 _ pe) ; 


Taking the limit a — 00, we find I*(a) > x. Therefore I = ,/t, as we found previ- 
ously. Substituting x = ./zy shows that the corresponding integral of exp(—ax?) 
has the value Jn/t. We use this result in the discussion of the normal distribution 
in chapter 30. 


6.4.3 Change of variables in triple integrals 


A change of variable in a triple integral follows the same general lines as that for 
a double integral. Suppose we wish to change variables from x, y, z to u, v, w. 
In the x, y, z coordinates the element of volume is a cuboid of sides dx, dy, dz 
and volume dV,,, = dx dy dz. If, however, we divide up the total volume into 
infinitesimal elements by constructing a grid formed from the coordinate surfaces 
u = constant, v = constant and w = constant, then the element of volume dYV,,,,, 
in the new coordinates will have the shape of a parallelepiped whose faces are the 
coordinate surfaces and whose edges are the curves formed by the intersections of 
these surfaces (see figure 6.12). Along the line element PQ the coordinates v and 
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w are constant, and so PQ has components (0x/0u) du, (6y/0u) du and (0z/0u) du 
in the directions of the x-, y- and z- axes respectively. The components of the 
line elements PS and ST are found by replacing u by v and w respectively. 

The expression for the volume of a parallelepiped in terms of the components 
of its edges with respect to the x-, y- and z-axes is given in chapter 7. Using this, 
we find that the element of volume in u,v, w coordinates is given by 


6(x, y,Z) 


Ai.) du dv dw, 


MWruyy = 


where the Jacobian of x, y, z with respect to u, v, w is a short-hand for a 3 x 3 


determinant: 
ox oy dz 
u 


0 Ou 
OX YZ) _ ox oy dz 
O(uv,w) | dv Ov av 

ox oy dz 


ow dw Ow 
So, in summary, the relationship between the elemental volumes in multiple 
integrals formulated in the two coordinate systems is given in Jacobian form by 


A(x, y,Z) 
O(u, v, W) 


and we can write a triple integral in either set of coordinates as 


t= fff tosszrdvavaz = fff ete. 


dx dy dz = du dv dw, 


A(x, y,2) 


ORR du dv dw. 


> Find an expression for a volume element in spherical polar coordinates, and hence calcu- 
late the moment of inertia about a diameter of a uniform sphere of radius a and mass M. 


Spherical polar coordinates r,0,¢ are defined by 
x=rsinOcos¢, y=rsin@sing, z=rcosé 


(and are discussed fully in chapter 10). The required Jacobian is therefore 


af. sin @ cos d sin @ sin d cos 0 
sie) rcos@cosf rcos@sing —rsin@ 
d(r, 0, p) —rsin@singd rsinOcos¢ 0 


The determinant is most easily evaluated by expanding it with respect to the last column 
(see chapter 8), which gives 


J = cos 0(r’ sin 6 cos 0) + r sin 0(r sin? 0) 
=r’ sin 0(cos” 6 + sin? 0) =r? sind. 


Therefore the volume element in spherical polar coordinates is given by 


_ (x,y,z) 2B 
ae CR a ee sin 0 dr dO dd, 
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which agrees with the result given in chapter 10. 
If we place the sphere with its centre at the origin of an x, y, z coordinate system then 
its moment of inertia about the z-axis (which is, of course, a diameter of the sphere) is 


r= f (+9?) am =p [ (2 +s) dV, 


where the integral is taken over the sphere, and p is the density. Using spherical polar 
coordinates, we can write this as 


r= fff (7? sin? 0) 7° sin 0 dr dO dd 
. 
2n Tt a 
=o [ ap [ ao sin* o [ dr r* 
0 1) 0 


= M1 Bad 
=pXx2nx 5X s@ = FIM p. 


Since the mass of the sphere is M = tna’, the moment of inertia can also be written as 
I =2Ma’. < 


6.4.4 General properties of Jacobians 


Although we will not prove it, the general result for a change of coordinates in 
an n-dimensional integral from a set x; to a set y; (where i and j both run from 
1 to n) is 

O(X1, X2,--+5Xn) 
O(V1, Y25+++5 Yn) 
where the n-dimensional Jacobian can be written as an n x n determinant (see 
chapter 8) in an analogous way to the two- and three-dimensional cases. 

For readers who already have sufficient familiarity with matrices (see chapter 8) 
and their properties, a fairly compact proof of some useful general properties 
of Jacobians can be given as follows. Other readers should turn straight to the 
results (6.16) and (6.17) and return to the proof at some later time. 

Consider three sets of variables x;, y; and z;, with i running from 1 to n for 
each set. From the chain rule in partial differentiation (see (5.17)), we know that 


oxi =o OX: OK (6.13) 


n 
Az; Ov, Oz: 
O25 Ta OV 02; 


dx, dxz+-++dxpn) = dy; dy2+++ dyn, 


Now let A, B and C be the matrices whose ijth elements are 0x;/0y;, 0y;/0z; and 
0x;/0z; respectively. We can then write (6.13) as the matrix product 


n 
Cij = Ss Aix dk j or C = AB. (6.14) 
k=1 


We may now use the general result for the determinant of the product of two 
matrices, namely |AB| = |A||B|, and recall that the Jacobian 


 A(V1y +665 Yn) 


O(X1,--+,Xn) 7 


= |Al, (6.15) 


xy 
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and similarly for Jy, and J,,. On taking the determinant of (6.14), we therefore 
obtain 


or, in the usual notation, 
O(X1,--+,Xn) = O(X1, ++ Xn) OV15+-+5 Yn) 
O(Z1,.++52n) O(V1,-- «5 Yn) O(Z1,-+-52n) 


As a special case, if the set z; is taken to be identical to the set x;, and the 
obvious result J, = 1 is used, we obtain 


(6.16) 


InyJyx = 1 
or, in the usual notation, 
Coen ae [orm | 
OW 15-++5 Yn) O(X1,..-5Xn) 


The similarity between the properties of Jacobians and those of derivatives is 
apparent, and to some extent is suggested by the notation. We further note from 


(6.17) 


(6.15) that since |A] = |A™|, where A’ is the transpose of A, we can interchange the 
rows and columns in the determinantal form of the Jacobian without changing 
its value. 


6.5 Exercises 


6.1 Identify the curved wedge bounded by the surfaces y? = 4ax, x +z = a and 
z = 0, and hence calculate its volume V. 

6.2 Evaluate the volume integral of x? + y? + z* over the rectangular parallelepiped 
bounded by the six surfaces x = +a, y = +b and z = tc. 

6.3 Find the volume integral of x?y over the tetrahedral volume bounded by the 
planes x = 0, y=0,2=0,andx+y4+z=1. 

6.4 Evaluate the surface integral of f(x,y) over the rectangleO<x<a,0<y<b 
for the functions 

x 
@feo= sa  O)fsy)=6-y +9. 
6.5, Calculate the volume of an ellipsoid as follows: 


(a) Prove that the area of the ellipse 
xy? 
ap 
is mab. 


(b) Use this result to obtain an expression for the volume of a slice of thickness 
dz of the ellipsoid 


rea Mea 


Hence show that the volume of the ellipsoid is 42abc/3. 
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6.6 


6.7 


6.8 


6.9 


6.10 


The function 


P(r) =A (2 = =") e228 


gives the form of the quantum-mechanical wavefunction representing the electron 
in a hydrogen-like atom of atomic number Z, when the electron is in its first 
allowed spherically symmetric excited state. Here r is the usual spherical polar 
coordinate, but, because of the spherical symmetry, the coordinates 0 and ¢ do 
not appear explicitly in ‘¥. Determine the value that A (assumed real) must have 
if the wavefunction is to be correctly normalised, ie. if the volume integral of 
||? over all space is to be equal to unity. 

In quantum mechanics the electron in a hydrogen atom in some particular state 
is described by a wavefunction Y, which is such that |‘P|? dV is the probability of 
finding the electron in the infinitesimal volume dV. In spherical polar coordinates 
W = V(r,0,) and dV =r’ sin 0 dr dO dd. Two such states are described by 


1/2 3/2 
a 
An do 


1/2 ; 3/2 1, ,—r/2a9 
Y= (=) sin @ e? (sn) Me ; 
8x 2ao ay/3 


(a) Show that each ; is normalised, i.e. the integral over all space f |‘¥|? dV is 
equal to unity — physically, this means that the electron must be somewhere. 

(b) The (so-called) dipole matrix element between the states 1 and 2 is given by 
the integral 


Px = J ¥iarsind cos Yo dV, 


where q is the charge on the electron. Prove that p, has the value —2’qao/3>. 


A planar figure is formed from uniform wire and consists of two equal semicircu- 
lar arcs, each with its own closing diameter, joined so as to form a letter ‘B’. The 
figure is freely suspended from its top left-hand corner. Show that the straight 
edge of the figure makes an angle 0 with the vertical given by tan@ = (24-2). 
A certain torus has a circular vertical cross-section of radius a centred on a 
horizontal circle of radius c (> a). 


(a) Find the volume V and surface area A of the torus, and show that they can 
be written as 


qn 


V=Ts—\ro—n),  A=e 0S — 1), 


where r, and r; are, respectively, the outer and inner radii of the torus. 
(b) Show that a vertical circular cylinder of radius c, coaxial with the torus, 
divides A in the ratio 


mc+2a : mc—2a. 
A thin uniform circular disc has mass M and radius a. 


(a) Prove that its moment of inertia about an axis perpendicular to its plane 
and passing through its centre is 3Ma’. 
(b) Prove that the moment of inertia of the same disc about a diameter is 3Ma’. 
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6.11 


6.12 


6.13 


6.14 


6.15 


6.16 


6.17 


This is an example of the general result for planar bodies that the moment of 
inertia of the body about an axis perpendicular to the plane is equal to the sum 
of the moments of inertia about two perpendicular axes lying in the plane; in an 
obvious notation 


t= [Pam= [o2+y\dm= fs2dm+ f ydm=1, +1. 


In some applications in mechanics the moment of inertia of a body about a 
single point (as opposed to about an axis) is needed. The moment of inertia, /, 
about the origin of a uniform solid body of density p is given by the volume 
integral 


I = [P+ y+ pav. 
Vv 


Show that the moment of inertia of a right circular cylinder of radius a, length 
2b and mass M about its centre is 


er Pp 
M{(~4+—-]}. 
The shape of an axially symmetric hard-boiled egg, of uniform density po, is 


given in spherical polar coordinates by r = a(2 — cos@), where 0 is measured 
from the axis of symmetry. 


(a) Prove that the mass M of the egg is M = Qi poa. 


; 5 anes wie be «342472 
(b) Prove that the egg’s moment of inertia about its axis of symmetry is 3 Ma’. 


In spherical polar coordinates r, 0, @ the element of volume for a body that 
is symmetrical about the polar axis is dV = 2zr? sin dr d0, whilst its element 
of surface area is 2zr sin 0[(dr)? + r?(d0)’]'/?. A particular surface is defined by 
r = 2acos@, where a is a constant and 0 < 0 < 7/2. Find its total surface area 
and the volume it encloses, and hence identify the surface. 

By expressing both the integrand and the surface element in spherical polar 
coordinates, show that the surface integral 


2 
/—=« 
x+y 


over the surface x? + y? = z?, 0 <z <1, has the value 2/,/2. 
By transforming to cylindrical polar coordinates, evaluate the integral 


r= ff fre +y*dxdyas 


over the interior of the conical region x? + y? < 2z?,0<z<1. 
Sketch the two families of curves 


y? = 4u(u— x), y? = 4v(v + x), 


where u and v are parameters. 

By transforming to the uv-plane, evaluate the integral of y/(x? + y?)'/? over 
the part of the quadrant x > 0, y > 0 that is bounded by the lines x = 0, y = 0 
and the curve y* = 4a(a — x). 

By making two successive simple changes of variables, evaluate 


r= ff [ eaxayas 
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6.18 


6.19 


6.20 


6.21 


6.22 


6.23 


over the ellipsoidal region 


Sketch the domain of integration for the integral 


1/y yp 
i=f es ~ expl w(x? + x7?)] dx dy 


and characterise its boundaries in terms of new variables u = xy and v = y/x. 
Show that the Jacobian for the change from (x, y) to (u,v) is equal to (2v)~', and 
hence evaluate I. 


Sketch the part of the region 0 < x, 0 < y < 2/2 that is bounded by the curves 
x =0, y =0, sinhxcos y = 1 and coshxsiny = 1. By making a suitable change 
of variables, evaluate the integral 


I = [ [tsinn? x + cos y) sinh 2x sin 2y dx dy 


over the bounded subregion. 
Define a coordinate system u,v whose origin coincides with that of the usual 
x,y system and whose u-axis coincides with the x-axis, whilst the v-axis makes 
an angle « with it. By considering the integral J = f exp(—r?) dA, where r is the 
radial distance from the origin, over the area defined by 0 <u<m,0<v0<@, 
prove that 

| | exp(—u? — v? — 2uv cosa) du dv = ee 

Oo <al0 2sin a 
As stated in section 5.11, the first law of thermodynamics can be expressed as 
dU =TdS —Pdv. 


By calculating and equating 6?>U/0Y0X and 0?U/0X0Y, where X and Y are an 
unspecified pair of variables (drawn from P,V,T and S), prove that 
OS,T)  O(V,P) 
O(X,Y) (X,Y) 
Using the properties of Jacobians, deduce that 
O(S,T) _ 
OV, P) 


The distances of the variable point P, which has coordinates x, y,z, from the fixed 
points (0,0, 1) and (0,0,—1) are denoted by u and v respectively. New variables 
é,n,@ are defined by 


f=Ztutv), 1=5(u-d), 


and ¢ is the angle between the plane y = 0 and the plane containing the three 
points. Prove that the Jacobian 0(é,1, @)/0(x, y,z) has the value (€? — n7)~! and 


that 
(u—v)? utv _ 16x 
/ / i space uv es ee 2 e ay a “3e/ 


This is a more difficult question about ‘volumes’ in an increasing number of 
dimensions. 


210 


6.6 HINTS AND ANSWERS 


6.1 


6.3 
6.5 


6.7 
6.9 


6.11 
6.13 
6.15 
6.17 
6.19 


6.21 
6.23 


(a) Let R be a real positive number and define K,, by 


R 
Ky = i, (R? — x?)" dx. 

Show, using integration by parts, that K,, satisfies the recurrence relation 
(2m + 1)Ky, = 2mR?K,,_1. 


(b) For integer n, define I, = K, and J, = Ky41/2. Evaluate Ip and Jo directly 
and hence prove that 


erly 1)? Ratt n(2n + 1) |R2nt2 
= ———_—_. and JI, =. 
(2n+ 1)! 2+InI(n + 1)! 
(c) A sequence of functions V,,(R) is defined by 
Vo(R) = 1, 


R 
V,(R) -/ Viet (VR = x) dx, n>. 
-R 


Prove by induction that 


qn" R2" i 22rtl | Rl 
Vong (R) = —————_ 
: an) (2n+ 1)! 


I, 


Von(R) = 


n! 
(d) For interest, 


(i) show that V2,42(1) < Vo, (1) and Van41(1) < V2n-1(1) for all n > 3; 

(ii) hence, by explicitly writing out V,(R) for 1 < k < 8 (say), show that the 
‘volume’ of the totally symmetric solid of unit radius is a maximum in 
five dimensions. 


6.6 Hints and answers 


For integration order z, y, x, the limits are (0,a — x), (—./4ax, ./4ax) and (0, a). 
For integration order y, x, z, the limits are (—./4ax, te ), (0,a — z) and (0, a). 
V = 1607/15. 

1/360. 

(a) Evaluate f 2b[1 — (x/a)*]'/? dx by setting x = acos ¢; 

(b) dV =x x afl —(z/ey]'? x bf — (z/c)?]!? dz. 

Write sin’ @ as (1 — cos” @) sin @ when integrating |P|?. 

(a) V = 2nc x na? and A = 2na x 2nc. Setting rp = c +a and r; = c —a gives the 
stated results. (b) Show that the centre of gravity of either half is 2a/z from the 
cylinder. 

Transform to cylindrical polar coordinates. 

4na?; 4na?/3; a sphere. 

The volume element is pd¢dpdz. The integrand for the final z-integration is 
given by 2n[(z7Inz) — (z*/2)]; 1 = —5x/9. 

Set € = x/a,y = y/b, € =z/c to map the ellipsoid onto the unit sphere, and then 
change from (é,7,) coordinates to spherical polar coordinates; I = 4na*bc/15. 
Set u = sinhxcosy and v = coshxsiny; Jyyw = (sinh? x + cos? y)-! and the 
integrand reduces to 4uv over the regionO<u<1,0<v<1;J=1. 

Terms such as Té?S/@Y 0X cancel in pairs. Use equations (6.17) and (6.16). 

(c) Show that the two expressions mutually support the integration formula given 
for computing a volume in the next higher dimension. 

(d)(ii) 2, 2, 42/3, 27/2, 827/15, 23/6, 1623/105, 24/24. 
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Vector algebra 


Scalars and vectors 


The simplest kind of physical quantity is one that can be completely specified by 
its magnitude, a single number, 

Such a quantity is called a scalar and examples include temperature, time and 
density. 

A vector is a quantity that requires both yd 9 eect 
space to specify it completely; we may think of 1t as an arrow in space. A tamiliar 
example is force, which has a magnitude (strength) measured in newtons and a 
direction of application. The large number of vectors that are used to describe 
the physical world include velocity, displacement, momentum and electric field. 
Vectors are also used to describe quantities such as angular momentum and 
surface elements (a surface element has an area and a direction defined by the 
normal to its tangent plane); in such cases their definitions may seem somewhat 
arbitrary (though in fact they are standard) and not as physically intuitive as for 
vectors such as force. : 


This chapter considers basic vector algebra and illustrates just how powerful 
vector analysis can be. All the techniques are presented for three-dimensional 
space but most can be readily extended to more dimensions. 

Throughout the book we will represent a vector in diagrams as a line together 
with an arrowhead. We will make no distinction between an arrowhead at the 
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7.2 ADDITION AND SUBTRACTION OF VECTORS 


at+b=b+a. 


The generalisation of this procedure to the addition of three (or more) vectors is 
clear oo 


a+(b+c)=(a+b)+e. 


Thus, it is immaterial in what order any number of vectors are added. 
The subtraction of two vectors is very similar to their addition (see figure 7.3), 
that is, 


a—b=a+(—b) 


where —b is a vector of equal magnitude but exactly opposite direction to vector b. 
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Cc 
b a 
a 
< 
b+e a+(b+c) 


Subtraction of two vectors. 


The subtraction of two equal vectors yields the zero vector, 0, which has zero 
magnitude and no associated direction. 


Multiplication by a scalar 


Multiplication of a vector by a scalar 

gives a vector in the same direction 
as the original but of a proportional magnitude. This can be seen in figure 7.4. 
The scalar may be positive, negative or zero. It can also be complex in some 
applications. Clearly, when the scalar is negative we obtain a vector pointing 
in the opposite direction to the original vector. Multiplication by a scalar is 
associative, commutative and distributive over addition. These properties may be 
summarised for arbitrary vectors a and b and arbitrary scalars 4 and yw by 


(Apa = A(ua) = p(Aa), 
2(a + b) = Aa + Ab, 
(A+ pa = dat pa. 


MULTIPLICATION BY A SCALAR 


Having defined the operations of addition, subtraction and multiplication by a 
scalar, we can now use vectors to solve simple problems in geometry. 


eA point P divides a line segment AB in the ratio 2 : u (see figure 7.5). If the position 
vectors of the points A and B are a and b, respectively, find the position vector of the 


point P. 


As is conventional for vector geometry problems, we denote the vector from the point A 
to the point B by AB. If the position vectors of the points A and B, relative to some origin 
O, are a and b, it should be clear that AB = b—a. 

Now, from figure 7.5 we see that one possible way of reaching the point P from O is 
first to go from O to A and to go along the line AB for a distance equal to the the fraction 
A/(A+ p) of its total length. We may express this in terms of vectors as 


OP = p=a+ ——AB 


which expresses the position vector of the point P in terms of those of A and B. We would, 
of course, obtain the same result by considering the path from O to B and then to P. < 
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Figure 7.6 The centroid of a triangle. The triangle is defined by the points A, 
B and C that have position vectors a, b and c. The broken lines CD, BE, AF 
connect the vertices of the triangle to the mid-points of the opposite sides; 
these lines intersect at the centroid G of the triangle. 


> The vertices of triangle ABC have position vectors a, b and ¢ relative to some origin O 
(see figure 7.6). Find the position vector of the centroid G of the triangle. 


From figure 7.6, the points D and E bisect the lines AB and AC respectively. Thus from 
the ratio theorem (7.6), with 2 = = 1/2, the position vectors of D and E relative to the 
origin are 
d= 5a + 3b, 
e= fat se. 
Using the ratio theorem again, we may write the position vector of a general point on the 
line CD that divides the line in the ratio 2 : (1 — 4) as 
r=(1—A)c+ Ad, 
= (1—A)e + 5A(a +b), (7.7) 
where we have expressed d in terms of a and b. Similarly, the position vector of a general 
point on the line BE can be expressed as 
r=(1—yp)b+ pe, 
= (1—p)b+ $u(a +e). (7.8) 
Thus, at the intersection of the lines CD and BE we require, from (7.7), (7.8), 
(1—Aje+ 5A(a +b) = (1 — wb + Su(at+e). 


By equating the coefficents of the vectors a, b, c we find 


A=U, sAa=l—w, 1-A=Su. 


216 


74 BASIS VECTORS AND COMPONENTS 


These equations are consistent and have the solution 2 = nw = 2/3. Substituting these 
values into either (7.7) or (7.8) we find that the position vector of the centroid G is given 
by 


g=t(at+b+e). < 


Basis vectors and components 


Given any three different vectors e;, e2 and e3, which do not all lie in a plane, 
it is possible, in a three-dimensional space, to write any other vector in terms of 
scalar multiples of them: 


a= aye; + doe) + 3e3. 


The three vectors e;, e) and e3 are said to form a basis (for the three-dimensional 
space); the scalars a,, a) and a3, which may be positive, negative or zero, are 
called the components of the vector a with respect to this basis. We say that the 
vector has been resolved into components. 

Most often we shall use basis vectors that are mutually perpendicular, for ease 
of manipulation, though this is not necessary. In general, a basis set must 


(i) have as many basis vectors as the number of dimensions (in more formal 
language, the basis vectors must span the space) and 

(ii) be such that no basis vector may be described as a sum of the others, or, 
more formally, the basis vectors must be linearly independent. Putting this 
mathematically, in N dimensions, we require 


cyey + C9@ +--+ + cyen #9, 
for any set of coefficients c1,c¢2,...,cy except c) = C2 =-+--=cy = 0. 


In this chapter we will only consider vectors in three dimensions; higher dimen- 
sionality can be achieved by simple extension. 

If we wish to label points in space using a Cartesian coordinate system (x, y, Z), 
we may introduce the unit vectors i, j and k, which point along the positive x-, 
y- and z- axes respectively. A vector a may then be written as a sum of three 
vectors, each parallel to a different coordinate axis: 


a = ayi+ ayjtazk. (7.10) 


A vector in three-dimensional space thus requires three components to describe 
fully both its direction and its magnitude. A displacement in space may be 
thought of as the sum of displacements along the x-, y- and z- directions (see 
figure 7.7). For brevity, the components of a vector a with respect to a particular 
coordinate system are sometimes written in the form (a,,a,,a,). Note that the 


217 


VECTOR ALGEBRA 


# ayi 


i 


Figure 7.7. A Cartesian basis set. The vector a is the sum of ai, a,j and a_k. 


basis vectors i, j and k may themselves be represented by (1,0,0), (0,1,0) and 
(0,0, 1) respectively. 

We can consider the addition and subtraction of vectors in terms of their 
components. The sum of two vectors a and b is found by simply adding their 
components, i.e. 


a+b = a,it+ a,j t+ ak + byi+ byjt+ b-k 
= (dy + by)i + (ay + by)j + (a, + bz)k, (7.11) 


and their difference by subtracting them, 


a—b=a,i+ a,j+a,k — (byi+ b,j + -k) 
= (ay — by)i + (ay — by)j + (a, — b.)k. (7.12) 


» Two particles have velocities y, = i+ 3j + 6k and vy. = i— 2k, respectively. Find the 
velocity u of the second particle relative to the first. 


The required relative velocity is given by 
u=vy—y =(1—1)i+ (0— 3)j + (—2 —6)k 
= —3j—8k. < 


7.5 Magnitude of a vector 


The magnitude of the vector a is denoted by |a| or a. In terms of its components 
in three-dimensional Cartesian coordinates, the magnitude of a is given by 


a= |a| = Jap +a + ae. (7.13) 


Hence, the magnitude of a vector is a measure of its length. Such an analogy is 
useful for displacement vectors but magnitude is better described, for example, by 
‘strength’ for vectors such as force or by ‘speed’ for velocity vectors. For instance, 
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ee 
bcos0 


Figure 7.8 The projection of b onto the direction of a is bcos 0. The scalar 
product of a and b is abcos 0. 


in the previous example, the speed of the second particle relative to the first is 
given by 


u= jul = (3)? + (8? = 73. 


A vector whose magnitude equals unity is called a unit vector. The unit vector 
in the direction a is usually notated 4 and may be evaluated as 


. a 
a=—. (7.14) 

la| 
The unit vector is a useful concept because a vector written as 1a then has mag- 
nitude 4 and direction 4. Thus magnitude and direction are explicitly separated. 


Multiplication of vectors 


We have already considered multiplying a vector by a scalar. Now we consider 
the concept of multiplying one vector by another vector. It is not immediately 
obvious what the product of two vectors represents and in fact two products 
are commonly defined, the scalar product and the vector product. As their names 
imply, the scalar product of two vectors is just a number, whereas the vector 
product is itself a vector. 


Scalar product 


The scalar product (or dot product) of two vectors a and b is denoted by a-b 
and is given by 


SKaler Garprnda 0<0<n, (7.15) 


where 0 is the angle between the two vectors, placed ‘tail to tail’ or ‘head to head’. 
Thus, the value of the scalar product a-b equals the magnitude of a multiplied 
by the projection of b onto a © 
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SKake Garpimdg 8" 


ayn deger el’ — 
t’e esiftir: Games») 


Examples of scalar products arise naturally throughout physics and in partic- 
ular in connection with energy. Perhaps the simplest is the work done F-r in 
moving the point of application of a constant force F through a displacement r; 
notice that, as expected, if the displacement is perpendicular to the direction of 
the force then F-r = 0 and no work is done. A second simple example is afforded 
by the potential energy —m- B of a magnetic dipole, represented in strength and 
orientation by a vector m, placed in an external magnetic field B. 

As the name implies, the scalar product has a magnitude but no direction. 
scalar product is commutative and distributive over addition: 


(Kale garpimin ence 
deg igme jell Vardi, ab te) =a btae 


d—a)-(c—b) =0. 


(d —b)- (ec—a) = 0. 


(d—a) -(c—b) = (d—b)- (c—a), 


d-c—a-c—d-b+a-b=d-c—b-c—d-a+b-a. 
Cancelling terms that appear on both sides and rearranging yields 
d-b—d-a—c-b+c-a=0O, 


which simplifies to give 
(d—c)-(b—a) =0. 


If we introduce a set of basis vectors that are mutually orthogonal, such as i, j, 
k, we can write the components of a vector a, with respect to that basis, in terms 
of the scalar product of a with each of the basis vectors, i.e. a, = a-i, ay = a-j and 
a, = a-k. In terms of the components ay, a, and a, the scalar product is given by 


a-b = (ayi+ ayj + ak) - (bi + byj + bzk) = ayby + ayby + azb:, (7.21) 


where the cross terms such as a,i- b,j are zero because the basis vectors are 
mutually perpendicular; see equation (7.18). It should be clear from (7.15) that 
the value of a- b has a geometrical definition and that this value is independent 
of the actual basis vectors used. 


> Find the angle between the vectors a =i+ 2j+ 3k and b = 2i+ 3j + 4k. 


From (7.15) the cosine of the angle @ between a and b is given by 


From (7.21) the scalar product a-b has the value 
a-b=1x24+2x3+4+3x4=20, 
and from (7.13) the lengths of the vectors are 


laj=JP4+2432=/14 and [bl = /224+3224+4 = 129. 
Thus, 


cos 0 = ees = 0.9926 > 0=0.12 rad. < 


/14,/29 
We can see from the expressions (7.15) and (7.21) for the scalar product that if 
0 is the angle between a and b then 


Oy Diy By «tae bs 
a b a b 7 ab 
where a,/a, ay/a and a,/a are called the direction cosines of a, since they give the 
cosine of the angle made by a with each of the basis vectors. Similarly b,/b, by/b 
and b,/b are the direction cosines of b. 

If we take the scalar product of any vector a with itself then clearly 0 = 0 and 
from (7.15) we have 


cos? = 


a-a=|al’. 


Thus the magnitude of a can be written in a coordinate-independent form as 


jal = /a-a. 


Finally, we note that the scalar product may be extended to vectors with 
complex components if it is redefined as 


a-b=ayby +a,by + azz, 


where the asterisk represents the operation of complex conjugation. To accom- 
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a 


Figure 7.9 The vector product. The vectors a, b and ax b form a right-handed 
set. 


modate this extension the commutation property (7.19) must be modified to 
read 


a-b=(b-a)’. (7.22) 


In particular it should be noted that (Aa). b = 4*a-b, whereas a- (Ab) = Aa- b. 
However, the magnitude of a complex vector is still given by |a| = ,/a-a, since 
a-a is always real. 


7.6.2 Vector product 


The vector product (or cross product) of two vectors a and b is denoted by a x b 
and is defined to be a vector of magnitude |a||b| sin 0 in a direction perpendicular 
to both a and b; 


ja x b| = |al|b| sin 0. 


The direction is found by ‘rotating’ a into b through the smallest possible angle. 
The sense of rotation is that of a right-handed screw that moves forward in the 
direction a x b (see figure 7.9). Again, @ is the angle between the two vectors 
placed ‘tail to tail’ or ‘head to head’. With this definition a, b and a x b form a 
right-handed set. A more directly usable description of the relative directions in 
a vector product is provided by a right hand whose first two fingers and thumb 
are held to be as nearly mutually perpendicular as possible. If the first finger is 
pointed in the direction of the first vector and the second finger in the direction 
of the second vector, then the thumb gives the direction of the vector product. 

The vector product is distributive over addition, but anticommutative and non- 
associative: 


(a+b) xc=(axc)+(bx ce), (7.23) 
bx a= -—/(a x b), (7.24) 
(ax b) xc #ax (bxc). (7.25) 
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Figure 7.10 The moment of the force F about O is r x F. The cross represents 
the direction of r x F, which is perpendicularly into the plane of the paper. 


From its definition, we see that the vector product has the very useful property 
that if a x b = 0 then a is parallel or antiparallel to b (unless either of them is 
zero). We also note that 


axa=0. (7.26) 


> Show that ifa =b-+ Ae, for some scalar 2, thenaxc=bx ce. 


axc=(b+/c)xc=bxet+iexe. 


However, from (7.26), e x ec = 0 and so 
axc=bxe. (7.27) 
We note in passing that the fact that (7.27) is satisfied does not imply that a= b. < 


An example of the use of the vector product is that of finding the area, A, of 
a parallelogram with sides a and b, using the formula 


A=|axbl. (7.28) 


Another example is afforded by considering a force F acting through a point R, 
whose vector position relative to the origin O is r (see figure 7.10). Its moment 
or torque about O is the strength of the force times the perpendicular distance 
OP, which numerically is just Fr sin 0, i.e. the magnitude of r x F. Furthermore, 
the sense of the moment is clockwise about an axis through O that points 
perpendicularly into the plane of the paper (the axis is represented by a cross 
in the figure). Thus the moment is completely represented by the vector r x F, 
in both magnitude and spatial sense. It should be noted that the same vector 
product is obtained wherever the point R is chosen, so long as it lies on the line 
of action of F. 

Similarly, if a solid body is rotating about some axis that passes through the 
origin, with an angular velocity w then we can describe this rotation by a vector 
q that has magnitude w and points along the axis of rotation. The direction of w 
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is the forward direction of a right-handed screw rotating in the same sense as the 
body. The velocity of any point in the body with position vector r is then given 
by v=o xr. 

Since the basis vectors i, j, k are mutually perpendicular unit vectors, forming 
a right-handed set, their vector products are easily seen to be 


brs ZEEE Vet Gorpinda oy 
ixj=-jxi=k, defelerin Gap 


jxk=—-kxj=i, 


kxi=-ixk=j. Sibirdie 


a x b = (ay. — azby)i + (aby — axbz)j + (axby — aybs)k. 


ij k 
axb=] a, dy a, 
by by bz 


> Find the area A of the parallelogram with sides a = i+ 2j+ 3k and b = 4i+ 5j-+ 6k. 


The vector product a x b is given in component form by 


axb=(2x6—3x 5)ji+(3 x 4-1 x 6)j+(1 x 5-2 x 4)k 
= —3i+ 6j — 3k. 


Thus the area of the parallelogram is 


A=|axbl = /-3"P +243) = /54. « 


[a,b,c] =a-(b x c) 


and, as its name suggests, it is just a number. It is most simply interpreted as the 
volume of a parallelepiped whose edges are given by a, b and c (see figure 7.11). 
The vector v = a x b is perpendicular to the base of the solid and has magnitude 
v = absin 90, i.e. the area of the base. Further, v- ¢ = vccos ¢. Thus, since ccos 
= OP is the vertical height of the parallelepiped, it is clear that (a x b)-c = area 
of the base x perpendicular height = volume. It follows that, if the vectors a, b 
and ¢ are coplanar, a- (b x c) = 0. 

Expressed in terms of the components of each vector with respect to the 
Cartesian basis set i, j, k the scalar triple product is 


a-(b Xx c) = ay(byc, — bzcy) + ay(bzex — byez) + az (bxcy — bycx), 


ay “aypods 
a-(bxc)=| by by bz 
eh Cy. nC 


By writing the vectors in component form, it can be shown that 
a-(b xc) =(axb)-ce, 


so that the dot and cross symbols can be interchanged without changing the result. 
More generally, the scalar triple product is unchanged under cyclic permutation 
of the vectors a,b,c. Other permutations simply give the negative of the original 
scalar triple product. These results can be summarised by 


[a, b,c] = [b,c,a] = [c, a,b] [a,c, b] = —[b, a,c] = —[e,b, a]. (7.35) 
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> Find the volume V of the parallelepiped with sides a = i+ 2j + 3k, b = 41+ 5j+ 6k and 
c= 7i+ 8j + 10k. 


We have already found that a x b = —3i-+ 6j — 3k, in subsection 7.6.2. Hence the volume 
of the parallelepiped is given by 


V =|a-(b x c)| = |(a x b)-e¢| 
= |(—3i + 6j — 3k) - (7i + 8j + 10k)| 
= |(—3)(7) + (6)(8) + (—3)(10)| = 3. < 


Another useful formula involving both the scalar and vector products is La- 
grange’s identity (see exercise 7.9), Le. 


(a x b)- (ec x d) = (a- e)(b- d) — (a- d)(b- ce). (7.36) 


7.6.4 Vector triple product 


By the vector triple product of three vectors a, b, c we mean the vector a x (b x c). 
Clearly, a x (b x ¢) is perpendicular to a and lies in the plane of b and c and so 
can be expressed in terms of them (see (7.37) below). We note, from (7.25), that 
the vector triple product is not associative, Le. a x (b x c) # (a x b) x ¢. 

Two useful formulae involving the vector triple product are 


a x (b x c) = (a- c)b — (a- b)e, (7.37) 
(a x b) x c = (a- c)b— (b- c)a, (7.38) 


which may be derived by writing each vector in component form (see exercise 7.8). 
It can also be shown that for any three vectors a, b, c, 


ax (bx c)+bx (cx a) +e x (a x b) = 0. 


7.7 Equations of lines, planes and spheres 


Now that we have described the basic algebra of vectors, we can apply the results 
to a variety of problems, the first of which is to find the equation of a line in 
vector form. 


7.7.1 Equation of a line 


Consider the line passing through the fixed point A with position vector a and 
having a direction b (see figure 7.12). It is clear that the position vector r of a 
general point R on the line can be written as 


r=a+t)b, (7.39) 
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Figure 7.12 The equation of a line. The vector b is in the direction AR and 
Ab is the vector from A to R. 


since R can be reached by starting from O, going along the translation vector 
a to the point A on the line and then adding some multiple Ab of the vector b. 
Different values of 4 give different points R on the line. 

Taking the components of (7.39), we see that the equation of the line can also 
be written in the form 


constant. (7.40) 


F az 
by by b: 


Taking the vector product of (7.39) with b and remembering that b x b = 0 gives 
an alternative equation for the line 


(r—a)xb=0. 


We may also find the equation of the line that passes through two fixed points 
A and C with position vectors a and c. Since AC is given by ec — a, the position 
vector of a general point on the line is 


r=a+(c—a). 


7.7.2 Equation of a plane 


The equation of a plane through a point A with position vector a and perpendic- 
ular to a unit position vector fi (see figure 7.13) is 


(r—a)- =O. (7.41) 


This follows since the vector joining A to a general point R with position vector 
ris r—a;r will lie in the plane if this vector is perpendicular to the normal to 
the plane. Rewriting (7.41) as r- fi = a- fi, we see that the equation of the plane 
may also be expressed in the form r- fi = d, or in component form as 


Ix + my +nz = d, (7.42) 
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Figure 7.13 The equation of the plane is (r —a)- fi = 0. 


A 


where the unit normal to the plane is h = li+mj+nk and d = a- fi is the 
perpendicular distance of the plane from the origin. 
The equation of a plane containing points a, b and c is 


r=a+A(b—a)+4+ “l(c — a). 


This is apparent because starting from the point a in the plane, all other points 
may be reached by moving a distance along each of two (non-parallel) directions 
in the plane. Two such directions are given by b—a and c —a. It can be shown 
that the equation of this plane may also be written in the more symmetrical form 


r=ga+fb+ye, 


where a+ fP+y=1. 


Find the direction of the line of intersection of the two planes x + 3y —z = 5 and 
2x —2y+4z =3. 


The two planes have normal vectors ny = i+ 3j —k and nm = 2i— 2j + 4k. It is clear 
that these are not parallel vectors and so the planes must intersect along some line. The 


direction p of this line must be parallel to both planes and hence perpendicular to both 
normals. Therefore 


p= Xm 


[(3)(4) — (—2)(—D) i + (D2) — G)(4)1 5 + ()(-2) — 3)(2)] k 
= 10i— 6j—8k. « 


7.7.3, Equation of a sphere 


Clearly, the defining property of a sphere is that all points on it are equidistant 
from a fixed point in space and that the common distance is equal to the radius 
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of the sphere. This is easily expressed in vector notation as 


lr—e/? =(r—c)- (r—c) =’, (7.43) 


where c is the position vector of the centre of the sphere and a is its radius. 


> Find the radius p of the circle that is the intersection of the plane fi-r = p and the sphere 
of radius a centred on the point with position vector c. 


The equation of the sphere is 

jr—e? =a’, (7.44) 
and that of the circle of intersection is 

r—b? =p, (7.45) 


where r is restricted to lie in the plane and b is the position of the circle’s centre. 

As b lies on the plane whose normal is fi, the vector b —e must be parallel to fi, ie. 
b—c = Jf for some 4. Further, by Pythagoras, we must have p? + |b — |? = a’. Thus 
P=a0—p’. 

Writing b = ¢ + \/a@? — pf and substituting in (7.45) gives 


r?—2r- (c+ Va pr) +242 Ve — peta — p? =p’, 
whilst, on expansion, (7.44) becomes 
Peete =a’. 
Subtracting these last two equations, using fi: r = p and simplifying yields 
p-—c-h= Jae — p. 


On rearrangement, this gives p as \/a* —(p—c-fi)*, which places obvious geometrical 
constraints on the values a,c,fi and p can take if a real intersection between the sphere 
and the plane is to occur. < 


7.8 Using vectors to find distances 


This section deals with the practical application of vectors to finding distances. 
Some of these problems are extremely cumbersome in component form, but they 
all reduce to neat solutions when general vectors, with no explicit basis set, 
are used. These examples show the power of vectors in simplifying geometrical 
problems. 


7.8.1 Distance from a point to a line 


Figure 7.14 shows a line having direction b that passes through a point A whose 
position vector is a. To find the minimum distance d of the line from a point P 
whose position vector is p, we must solve the right-angled triangle shown. We see 
that d = |p—al sin 0; so, from the definition of the vector product, it follows that 


d= |(p—a) x Bl. 


229 


VECTOR ALGEBRA 


0) 


Figure 7.14 The minimum distance from a point to a line. 


> Find the minimum distance from the point P with coordinates (1,2, 1) to the line r = a+Ab, 
where a=i+j+k and b = 2i—j+ 3k. 


Comparison with (7.39) shows that the line passes through the point (1,1,1) and has 
direction 2i—j-+ 3k. The unit vector in this direction is 


b= + oi-j 438). 


Ji4 


The position vector of P is p=i+ 2j+k and we find 


ginal 
(p—a) x b= To [jx 2i- 3) + 3b] 
ai 515 


Thus the minimum distance from the line to the point P is d = \/13/14. < 


7.8.2 Distance from a point to a plane 


The minimum distance d from a point P whose position vector is p to the plane 
defined by (r — a)- fi = 0 may be deduced by finding any vector from P to the 
plane and then determining its component in the normal direction. This is shown 
in figure 7.15. Consider the vector a — p, which is a particular vector from P to 
the plane. Its component normal to the plane, and hence its distance from the 
plane, is given by 


d=(a—p)-f, (7.46) 
where the sign of d depends on which side of the plane P is situated. 
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Figure 7.15 The minimum distance d from a point to a plane. 


> Find the distance from the point P with coordinates (1,2,3) to the plane that contains the 
points A, B and C having coordinates (0, 1,0), (2,3, 1) and (5,7, 2). 


Let us denote the position vectors of the points A, B, C by a, b, c. Two vectors in the 
plane are 
b—a=2i+2j+k and c—a=S5i+ 6j+ 2k, 
and hence a vector normal to the plane is 
n= (2i+ 2j+k) x (Si+ 6j+ 2k) = —2i+ j + 2k, 
and its unit normal is 
i= a = 1(—21+ j+ 2k). 


Denoting the position vector of P by p, the minimum distance from the plane to P is 
given by 


= (-i—j—3k)- }(-21+j+ 2k) 


2 5 
= -3. 


If we take P to be the origin O, then we find d = i ie. a positive quantity. It follows from 
this that the original point P with coordinates (1,2,3), for which d was negative, is on the 
opposite side of the plane from the origin. < 


7.8.3 Distance from a line to a line 


Consider two lines in the directions a and b, as shown in figure 7.16. Since a x b 
is by definition perpendicular to both a and b, the unit vector normal to both 
these lines is 
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O 


Figure 7.16 The minimum distance from one line to another. 


If p and q are the position vectors of any two points P and Q on different lines 
then the vector connecting them is p—q. Thus, the minimum distance d between 
the lines is this vector’s component along the unit normal, i.e. 


d=(|(p—q)- fi. 


>A line is inclined at equal angles to the x-, y- and z-axes and passes through the origin. 
Another line passes through the points (1,2,4) and (0,0,1). Find the minimum distance 
between the two lines. 


The first line is given by 


r, =A(it+j+k), 
and the second by 
rm =k+ “(it 2j+ 3k). 
Hence a vector normal to both lines is 
n= (i+j+k) x (i+ 2j)+ 3k) =i—2j+k, 
and the unit normal is 


1 
h = —(i—2j+k). 
TB! j+k) 
A vector between the two lines is, for example, the one connecting the points (0,0, 0) 


and (0,0,1), which is simply k. Thus it follows that the minimum distance between the 
two lines is 


1 
d= —|k- (i—2j+k)| = < 
yen! j+k)| 


al- 


7.8.4 Distance from a line to a plane 


Let us consider the line r = a+ Ab. This line will intersect any plane to which it 
is not parallel. Thus, if a plane has a normal fi then the minimum distance from 
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the line to the plane is zero unless 
b-ii=0, 
in which case the distance, d, will be 
d= |\(a—r)- fil, 


where r is any point in the plane. 


PA line is given by r = a+ Ab, where a = i+ 2j + 3k and b = 41 + 5j+ 6k. Find the 
coordinates of the point P at which the line intersects the plane 


x+2y+3z =6. 


A vector normal to the plane is 
n=i+ 2j+ 3k, 
from which we find that b- n # 0. Thus the line does indeed intersect the plane. To find 


the point of intersection we merely substitute the x-, y- and z- values of a general point 
on the line into the equation of the plane, obtaining 


14424212454) +3(3 +64) =6 144324 =6. 


This gives 2 = —t, which we may substitute into the equation for the line to obtain 


x =1—4}(4) =0, y =2—4(5) = 3 and z = 3— 4(6) = 3. Thus the point of intersection is 


7.9 Reciprocal vectors 


The final section of this chapter introduces the concept of reciprocal vectors, 
which have particular uses in crystallography. 
The two sets of vectors a, b, c and a’, b’, ce’ are called reciprocal sets if 


a-a’=b-b’=c-c'=1 (7.47) 
and 


a’- b=a'-c=b'-a=b'-c=c'-a=c'-b=0. (7.48) 


It can be verified (see exercise 7.19) that the reciprocal vectors of a, b and ¢ are 
given by 


bxe 


a = bce) (7.49) 
, cxa 

re eat (7.50) 
j axhb 


where a- (b x c) #0. In other words, reciprocal vectors only exist if a, b and ¢ are 
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not coplanar. Moreover, if a, b and ¢ are mutually orthogonal unit vectors then 
a’ =a, b’ = b and ce’ =¢, so that the two systems of vectors are identical. 


> Construct the reciprocal vectors of a = 2i,b =j +k, c=i+k. 


First we evaluate the triple scalar product: 
a-(b x ce) = 2i- [J +k) x G+k)] 
= 2i-(i+j—k) = 2. 
Now we find the reciprocal vectors: 
a =3(j+k) x (itk) = 5(+j—h, 
b’ = 5(i+k) x 2i = j, 
e = 402i) x Gtk) = -j+k 
It is easily verified that these reciprocal vectors satisfy their defining properties (7.47), 


(7.48). < 


We may also use the concept of reciprocal vectors to define the components of a 
vector a with respect to basis vectors e1, e2, e3 that are not mutually orthogonal. 
If the basis vectors are of unit length and mutually orthogonal, such as the 
Cartesian basis vectors i, j, k, then (see the text preceeding (7.21)) the vector a 
can be written in the form 


a=(a-iit(a-pjt(a-kk. 


If the basis is not orthonormal, however, then this is no longer true. Nevertheless, 
we may write the components of a with respect to a non-orthonormal basis 
ei, €2, €3 in terms of its reciprocal basis vectors e}, e, e;, which are defined as in 
(7.49)-(7.51). If we let 


a= aye; + deo + 0303, 
then the scalar product a-e{ is given by 
a-e; = aye; -e} + arene, + 43e3-e) = a1, 
where we have used the relations (7.48). Similarly, a7 = a-e, and a3 = a-e; so now 


a =(a-e})e; + (a- e)eo + (a: e4)es. (7.52) 


7.10 Exercises 


7A Which of the following statements about general vectors a, b and ¢ are true? 
) e-(ax b) =(bxa)-c. 

a x (bx c) =(a xb) xe. 

a x (b x c) = (a- e)b — (a: bec. 

d = ja + ub implies (a x b)-d =0. 

e) axc=bxc implies c-a—c-b =c\a—Dl. 

f) (ax b) x (ce x b) = bib. (ce x a). 
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7.2 


7.3 


74 


7.5 


7.6 


7.7 


78 


7.9 


7.10 


71 


A unit cell of diamond is a cube of side A, with carbon atoms at each corner, at 
the centre of each face and, in addition, at positions displaced by 5A(i +j+k) 
from each of those already mentioned; i, j, k are unit vectors along the cube axes. 
One corner of the cube is taken as the origin of coordinates. What are the vectors 
joining the atom at AC +j+k) to its four nearest neighbours? Determine the 
angle between the carbon bonds in diamond. 

Identify the following surfaces: 


(a) |r| =k; (b) r-u=/; (c)r-u=mlr| for-l1<m<-+l; 
(d) |r—(r-w)u| =n. 


Here k, |, m and n are fixed scalars and u is a fixed unit vector. 

Find the angle between the position vectors to the points (3, —4,0) and (—2, 1,0) 
and find the direction cosines of a vector perpendicular to both. 

A,B,C and D are the four corners, in order, of one face of a cube of side 2 
units. The opposite face has corners E,F,G and H, with AE, BF,CG and DH 
as parallel edges of the cube. The centre O of the cube is taken as the origin 
and the x-, y- and z-axes are parallel to AD, AE and AB, respectively. Find the 
following: 


(a) the angle between the face diagonal AF and the body diagonal AG; 

(b) the equation of the plane through B that is parallel to the plane CGE; 

(c) the perpendicular distance from the centre J of the face BCGF to the plane 
OCG; 

(d) the volume of the tetrahedron JOCG. 

Use vector methods to prove that the lines joining the mid-points of the opposite 


edges of a tetrahedron OABC meet at a point and that this point bisects each of 
the lines. 

The edges OP, OQ and OR of a tetrahedron OPQR are vectors p, q and r, 
respectively, where p = 2i+ 4j, q = 2i—j+ 3k and r = 4i — 2j + 5k. Show that 
OP is perpendicular to the plane containing OQR. Express the volume of the 
tetrahedron in terms of p, q and r and hence calculate the volume. 

Prove, by writing it out in component form, that 


(a x b) x c= (a-c)b—(b-c)a, 


and deduce the result, stated in equation (7.25), that the operation of forming 
the vector product is non-associative. 
Prove Lagrange’s identity, i.e. 


(a x b)- (ec x d) = (a-c)(b- d) — (a- d)(b-c). 
For four arbitrary vectors a, b, c and d, evaluate 
(a x b) x (ec x d) 
in two different ways and so prove that 
a[b, c,d] — b[c,d, a] + c[d, a, b] — d[a, b,c] = 0. 


Show that this reduces to the normal Cartesian representation of the vector d, 
Le. d,i+d,j+d_k, if a,b and ¢ are taken as i, j and k, the Cartesian base vectors. 
Show that the points (1,0, 1), (1, 1,0) and (1, —3, 4) lie on a straight line. Give the 
equation of the line in the form 


r=a+Ab. 


235 


7.12 


7.13 
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7.15 


7.16 


TAT 


7.18 


The plane P; contains the points A, B and C, which have position vectors 
a = —3i+ 2j, b = 7i+ 2j and c = 2i+ 3j+ 2k, respectively. Plane P) passes 
through A and is orthogonal to the line BC, whilst plane P; passes through 
B and is orthogonal to the line AC. Find the coordinates of r, the point of 
intersection of the three planes. 

Two planes have non-parallel unit normals f and m and their closest distances 
from the origin are 2 and y, respectively. Find the vector equation of their line 
of intersection in the form r = vp +a. 

Two fixed points, A and B, in three-dimensional space have position vectors a 
and b. Identify the plane P given by 


(a—b)-r = 4(a?—b’), 


where a and b are the magnitudes of a and b. 
Show also that the equation 


(a—r)-(b—r) =0 


describes a sphere S of radius |a —b|/2. Deduce that the intersection of P and S 
is also the intersection of two spheres, centred on A and B, and each of radius 
ja —b|/,/2. 

Let O, A, B and C be four points with position vectors 0, a, b and c, and denote 
by g = Aa+ ub+ ve the position of the centre of the sphere on which they all lie. 


(a) Prove that 4, » and v simultaneously satisfy 
(a-a)A+(a-b)ut+(a-eyv = 5a 


and two other similar equations. 

(b) By making a change of origin, find the centre and radius of the sphere on 
which the points p = 3i+j—2k, q = 41+ 3j—3k, r = 7i—3k ands = 6i+,j—k 
all lie. 


The vectors a, b and ¢ are coplanar and related by 
a+ ub+ve = 0, 


where 4, uu, v are not all zero. Show that the condition for the points with position 
vectors aa, Pb and yc to be collinear is 

A v 

tye yt ao, 

a poy 
Using vector methods: 


(a) Show that the line of intersection of the planes x + 2y + 3z = 0 and 
3x + 2y +z =0 is equally inclined to the x- and z-axes and makes an angle 
cos~!(—2/,/6) with the y-axis. 

(b) Find the perpendicular distance between one corner of a unit cube and the 
major diagonal not passing through it. 

Four points X;, i = 1,2,3,4, taken for simplicity as all lying within the octant 

x,y,z = 0, have position vectors x;. Convince yourself that the direction of 

vector x, lies within the sector of space defined by the directions of the other 


three vectors if 
. Xj ° Xj 
min , 
over j |xi[]xj| 


considered for i = 1,2,3,4 in turn, takes its maximum value for i = n, i.e. n equals 
that value of i for which the largest of the set of angles which x; makes with 
the other vectors, is found to be the lowest. Determine whether any of the four 
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7.19 


7.20 


7.21 


Figure 7.17 A face-centred cubic crystal. 


points with coordinates 
X = (3,2,2),  X2 = (23,1), X3 = (2,1,3), Xa = (3,0,3) 


lies within the tetrahedron defined by the origin and the other three points. 


The vectors a, b and ¢ are not coplanar. The vectors a’, b’ and ec’ are the 
associated reciprocal vectors. Verify that the expressions (7.49)-(7.51) define a set 
of reciprocal vectors a’, b’ and ce’ with the following properties: 


(a) a'-a=b'-b=c’-c=1; 

(b) a'- b=a'-c=b'-a_ etc=0; 
(c) [a’,b’,c] = 1/fa,b, el; 

(d) a=(b! x c’)/[a’,b’,c’]. 


Three non-coplanar vectors a, b and c, have as their respective reciprocal vectors 
the set a’, b’ and ec’. Show that the normal to the plane containing the points 
ka, I-'b and m~'c is in the direction of the vector ka’ + Ib’ + me’. 

In a crystal with a face-centred cubic structure, the basic cell can be taken as a 
cube of edge a with its centre at the origin of coordinates and its edges parallel 
to the Cartesian coordinate axes; atoms are sited at the eight corners and at the 
centre of each face. However, other basic cells are possible. One is the rhomboid 
shown in figure 7.17, which has the three vectors b, c and d as edges. 


(a) Show that the volume of the rhomboid is one-quarter that of the cube. 

(b) Show that the angles between pairs of edges of the rhomboid are 60° and that 
the corresponding angles between pairs of edges of the rhomboid defined by 
the reciprocal vectors to b, c, d are each 109.5°. (This rhomboid can be used 
as the basic cell of a body-centred cubic structure, more easily visualised as 
a cube with an atom at each corner and one at its centre.) 

(c) In order to use the Bragg formula, 2d sin 0 = nd, for the scattering of X-rays 
by a crystal, it is necessary to know the perpendicular distance d between 
successive planes of atoms; for a given crystal structure, d has a particular 
value for each set of planes considered. For the face-centred cubic structure 
find the distance between successive planes with normals in the k, i+j and 
i+j+k directions. 
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7.22 


7.23 


7.24 


In subsection 7.6.2 we showed how the moment or torque of a force about an axis 
could be represented by a vector in the direction of the axis. The magnitude of 
the vector gives the size of the moment and the sign of the vector gives the sense. 
Similar representations can be used for angular velocities and angular momenta. 


(a) The magnitude of the angular momentum about the origin of a particle of 
mass m moving with velocity v on a path that is a perpendicular distance d 
from the origin is given by m|v|d. Show that if r is the position of the particle 
then the vector J =r x mv represents the angular momentum. 

(b) Now consider a rigid collection of particles (or a solid body) rotating about 
an axis through the origin, the angular velocity of the collection being 
represented by w. 


(i) Show that the velocity of the ith particle is 
Vj = Xft; 


and that the total angular momentum J is 


J= y mi[re@ — (r+ @)ri]. 


(ii) Show further that the component of J along the axis of rotation can 
be written as Iw, where I, the moment of inertia of the collection 
about the axis or rotation, is given by 


I= mip?. 


Interpret p; geometrically. 
(iii) Prove that the total kinetic energy of the particles is s1@’. 


By proceeding as indicated below, prove the parallel axis theorem, which states 
that, for a body of mass M, the moment of inertia J about any axis is related to 
the corresponding moment of inertia Jy about a parallel axis that passes through 
the centre of mass of the body by 


I =I) + Maj, 


where a, is the perpendicular distance between the two axes. Note that Jo can 
be written as 


[x0 Gxnydm, 


where r is the vector position, relative to the centre of mass, of the infinitesimal 
mass dm and fi is a unit vector in the direction of the axis of rotation. Write a 
similar expression for I in which r is replaced by r’ = r —a, where a is the vector 
position of any point on the axis to which J refers. Use Lagrange’s identity and 
the fact that { rdm = 0 (by the definition of the centre of mass) to establish the 
result. 

Without carrying out any further integration, use the results of the previous 
exercise, the worked example in subsection 6.3.4 and exercise 6.10 to prove that 
the moment of inertia of a uniform rectangular lamina, of mass M and sides a 
and b, about an axis perpendicular to its plane and passing through the point 
(aa/2, bb/2), with -1 <a, f <1, is 


aaa + 302) + (1 + 382)]. 
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Figure 7.18 An oscillatory electric circuit. The power supply has angular 
frequency w = 2nf = 4007s". 


7.25 Define a set of (non-orthogonal) base vectors a=j+k,b=i+k ande =i+j. 


(a) Establish their reciprocal vectors and hence express the vectors p = 3i—2j+k, 
q =i+4j and r = —2i+-j+ k in terms of the base vectors a, b and ec. 

(b) Verify that the scalar product p-q has the same value, —5, when evaluated 
using either set of components. 


7.26 Systems that can be modelled as damped harmonic oscillators are widespread; 
pendulum clocks, car shock absorbers, tuning circuits in television sets and radios, 
and collective electron motions in plasmas and metals are just a few examples. 

In all these cases, one or more variables describing the system obey(s) an 
equation of the form 


X + 2px + whx = P cos at, 
where x = dx/dt, etc. and the inclusion of the factor 2 is conventional. In the 


steady state (i.e. after the effects of any initial displacement or velocity have been 
damped out) the solution of the equation takes the form 


x(t) = Acos(wt + ¢). 


By expressing each term in the form B cos(wt+e), and representing it by a vector 
of magnitude B making an angle e with the x-axis, draw a closed vector diagram, 
at t = 0, say, that is equivalent to the equation. 


(a) Convince yourself that whatever the value of wm (> 0) ¢ must be negative 


(—2 < @ < 0) and that 
—2yo 
= tan |. ——_—-], 
gat Gea) 


(b) Obtain an expression for A in terms of P, wo and . 


727 According to alternating current theory, the currents and potential differences in 
the components of the circuit shown in figure 7.18 are determined by Kirchhoff’s 
laws and the relationships 


Vi I V2 
R, 2= 2 
The factor i = ./—1 in the expression for I; indicates that the phase of J; is 90° 
ahead of V3. Similarly the phase of V4 is 90° ahead of I. 

Measurement shows that V3; has an amplitude of 0.661V) and a phase of 
+13.4° relative to that of the power supply. Taking Vo = 1 V, and using a series 


I; 


T3 iwCV3, V4 => i@Ll. 
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7.15 


TAT 


7.19 


7.21 


7.23 
7.25 


7.27 


of vector plots for potential differences and currents (they could all be on the 
same plot if suitable scales were chosen), determine all unknown currents and 
potential differences and find values for the inductance of L and the resistance 
of R». 

[Scales of 1cm = 0.1 V for potential differences and 1cm = 1 mA for currents 
are convenient. | 


7.11 Hints and answers 


(c), (d) and (e). 
(a) A sphere of radius k centred on the origin; (b) a plane with its normal in the 
direction of u and at a distance / from the origin; (c) a cone with its axis parallel 
to u and of semiangle cos! m; (d) a circular cylinder of radius n with its axis 
parallel to u. 
(a) cos“! \/2/3; (b) z — x = 2; (c) 1/,/2; (d) sale x8) i= 5. 
Show that q x r is sre to p; volume = 3 [$(q x r)-p] = 
Note that (a x b)- (¢ x d) =d- [(a x b) x ec] and use the real for a triple vector 
product to expand the expression in square brackets. 
Show that the position vectors of the points are linearly dependent; r = a+ /b 
where a=i+k and b=-j+k. 
Show that p must have the direction fix m and write a as xi+ ym. By obtaining a 
pair of simultaneous equations for x and y, prove that x = (A—pi-th)/[1—(f-m)?] 
and that y = (u— Af- m)/[1 — (f- m)’]. 
(a) Note that |ja—g|? = R* = |0—g’, leading toa-a = 2a-g 
(b) Make p the new origin and solve the three simultaneous linear equations to 

obtain 2 = 5/18, « = 10/18, v 3/18, giving g = 2i—k and a sphere of 

radius J centred on (5, 1,—3). 
(a) Find two points on both planes, say (0, 0,0) and (1,—2, 1), and hence determine 
the direction cosines of the line of intersection; (b) (4)!/?. 
For (c) and (d), treat (e x a) x (a x b) as a triple vector product with c x a as one 
of the three vectors. 
(b) bh) = a (i+ j+k), ¢ =a" (i-j+ bk), d = a! (i+, j—k); (c) a/2 for direction 
k; successive planes through (0,0,0) and (a/2,0,a/2) give a spacing of a/J/8 for 
direction i+j; successive planes through (—a/2,0,0) and (a/2,0,0) give a spacing 
of a/./3 for direction i +i +k. 
Note that a? — (fi- a)’ =a). 
p = —2a+ 3b, q = 3a— 3b+3 3c and r = 2a—b—c. Remember that a-a = b-b= 
ces 2andaboa coh: Dead 
With currents in mA and potential differences in volts: 
a (7.76, —23.2°), In = (14.36, —50.8°), I3 sue 30, 103.4°); 

= (0.388, —23.2°), Vz = (0.287, —50.8°), Vs = (0.596, 39.2°); 
= 33 mH, Ry = 20 Q. 
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In the previous chapter we defined a vector as a geometrical object which has 
both a magnitude and a direction and which may be thought of as an arrow fixed 
in our familiar three-dimensional space, a space which, if we need to, we define 
by reference to, say, the fixed stars. This geometrical definition of a vector is both 
useful and important since it is independent of any coordinate system with which 
we choose to label points in space. 

In most specific applications, however, it is necessary at some stage to choose 
a coordinate system and to break down a vector into its component vectors in 
the directions of increasing coordinate values. Thus for a particular Cartesian 
coordinate system (for example) the component vectors of a vector a will be ayi, 
a,j and a,k and the complete vector will be 


a = dyi+ ayjtazk. 


Although we have so far considered only real three-dimensional space, we may 
extend our notion of a vector to more abstract spaces, which in general can 
have an arbitrary number of dimensions N. We may still think of such a vector 
as an ‘arrow’ in this abstract space, so that it is again independent of any (N- 
dimensional) coordinate system with which we choose to label the space. As an 
example of such a space, which, though abstract, has very practical applications, 
we may consider the description of a mechanical or electrical system. If the state 
of a system is uniquely specified by assigning values to a set of N variables, 
which could be angles or currents, for example, then that state can be represented 
by a vector in an N-dimensional space, the vector having those values as its 
components. 

In this chapter we first discuss general vector spaces and their properties. We 
then go on to discuss the transformation of one vector into another by a linear 
operator. This leads naturally to the concept of a matrix, a two-dimensional array 
of numbers. The properties of matrices are then discussed and we conclude with 
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a discussion of how to use these properties to solve systems of linear equations. 
The application of matrices to the study of oscillations in physical systems is 
taken up in chapter 9. 


Vector spaces 
A set of objects (vectors) a, b, c, ... is said to form a linear vector space V if: 
(i) the set is closed under commutative and associative addition, so that 
a+b=b+a, (8.2) 
(a+b) +c=a+(b+c); (8.3) 


(ii) the set is closed under multiplication by a scalar (any complex number) to 
form a new vector Ja, the operation being both distributive and associative 


so that 
2(a + b) = Aa + Ab, (8.4) 
(A+ wa = a+ pa, (8.5) 
A(ua) = (Awa, (8.6) 


where / and yu are arbitrary scalars; 

(iii) there exists a null vector 0 such that a+ 0 =a for alla; 

(iv) multiplication by unity leaves any vector unchanged, ie. 1 x a =a; 

(v) all vectors have a corresponding negative vector —a such that a+(—a) = 0. 
It follows from (8.5) with 4 = 1 and w = —1 that —a is the same vector as 
(—1) xa. 


We note that if we restrict all scalars to be real then we obtain a real vector 
space (an example of which is our familiar three-dimensional space); otherwise, 
in general, we obtain a complex vector space. We note that it is common to use the 
terms ‘vector space’ and ‘space’, instead of the more formal ‘linear vector space’. 

The span of a set of vectors a,b,...,8 is defined as the set of all vectors that 
may be written as a linear sum of the original set, i.e. all vectors 


x=aa+fb+---+oas (8.7) 
that result from the infinite number of possible values of the (in general complex) 
scalars «, B,...,0. If x in (8.7) is equal to 0 for some choice of a, B,...,¢ (not all 
zero), Le. if 

ga+fb+---+os=0, (8.8) 


then the set of vectors a,b,...,s, is said to be linearly dependent. In such a set 
at least one vector is redundant, since it can be expressed as a linear sum of 
the others. If, however, (8.8) is not satisfied by any set of coefficients (other than 
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the trivial case in which all the coefficients are zero) then the vectors are linearly 
independent, and no vector in the set can be expressed as a linear sum of the 
others. 

If, in a given vector space, there exist sets of N linearly independent vectors, 
but no set of N + 1 linearly independent vectors, then the vector space is said to 
be N-dimensional. (In this chapter we will limit our discussion to vector spaces of 
finite dimensionality; spaces of infinite dimensionality are discussed in chapter 17.) 


8.1.1 Basis vectors 


If V is an N-dimensional vector space then any set of N linearly independent 
vectors €;,€2,...,en forms a basis for V. If x is an arbitrary vector lying in V then 
the set of N +1 vectors x,e1,€2,...,ey, must be linearly dependent and therefore 
such that 


ae; + Ber +---+o0en + xx = 0, (8.9) 
where the coefficients «, f,...,7 are not all equal to 0, and in particular x # 0. 
Rearranging (8.9) we may write x as a linear sum of the vectors e; as follows: 


N 
X= x1e1 + X2€2 +--+ xen = D_ xiei, (8.10) 
i=1 


for some set of coefficients x; that are simply related to the original coefficients, 
e.g. x1 = —a/y, X2 = —f/y, etc. Since any x lying in the span of V can be 
expressed in terms of the basis or base vectors e;, the latter are said to form 
a complete set. The coefficients x; are the components of x with respect to the 
e;-basis. These components are unique, since if both 


N N 
x= Se Xi@; and x= ye Viei, 
i=1 i=1 
then 
N 
So(xi — vie: = 0, (8.11) 
i=l 
which, since the e; are linearly independent, has only the solution x; = y; for all 
i=1,2,...,N. 
From the above discussion we see that any set of N linearly independent 


vectors can form a basis for an N-dimensional space. If we choose a different set 
e!,i=1,...,N then we can write x as 


N 
x=xje)txjeste--+txyey = y xie}. (8.12) 
i=l 
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We reiterate that the vector x (a geometrical entity) is independent of the basis 
— it is only the components of x that depend on the basis. We note, however, 
that given a set of vectors W,Ub,...,Uy, where M # N, in an N-dimensional 
vector space, then either there exists a vector that cannot be expressed as a 
linear combination of the u; or, for some vector that can be so expressed, the 
components are not unique. 


8.1.2 The inner product 


We may usefully add to the description of vectors in a vector space by defining 
the inner product of two vectors, denoted in general by (a|b), which is a scalar 
function of a and b. The scalar or dot product, a-b = |a||b| cos 0, of vectors 
in real three-dimensional space (where 0 is the angle between the vectors), was 
introduced in the last chapter and is an example of an inner product. In effect the 
notion of an inner product (a|b) is a generalisation of the dot product to more 
abstract vector spaces. Alternative notations for (a|b) are (a,b), or simply a- b. 
The inner product has the following properties: 


(i) (alb) = (bla)”, 
(ii) (a|Ab + pc) = A(alb) + p(ale). 


We note that in general, for a complex vector space, (i) and (1i) imply that 


(Aa + ble) = 2* (ale) + ne" (ble), (8.13) 
(Aa|ub) = 2° u(alb). (8.14) 


Following the analogy with the dot product in three-dimensional real space, 
two vectors in a general vector space are defined to be orthogonal if (a|b) = 0. 
Similarly, the norm of a vector a is given by |jal| = (ala)!/? and is clearly a 
generalisation of the length or modulus |a| of a vector a in three-dimensional 
space. In a general vector space (ala) can be positive or negative; however, we 
shall be primarily concerned with spaces in which (ala) > 0 and which are thus 
said to have a positive semi-definite norm. In such a space (ala) = 0 implies a = 0. 

Let us now introduce into our N-dimensional vector space a basis €1, @,...,éy 
that has the desirable property of being orthonormal (the basis vectors are mutually 
orthogonal and each has unit norm), i.e. a basis that has the property 


(@i|@;) = dij. (8.15) 


Here 6,; is the Kronecker delta symbol (of which we say more in chapter 26) and 


has the properties 
1 fori=j, 
Oi = Ldn 
0 fori F# j. 
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In the above basis we may express any two vectors a and b as 


N N 
a= ; ae; and b= ) bjé. 
i=1 i=1 


Furthermore, in such an orthonormal basis we have, for any a, 


N 


(@jla) = )\(ejlaiéi) = at 6,I8)) = aj. (8.16) 
i=1 


i=l 
Thus the components of a are given by a; = (@|a). Note that this is not true 


unless the basis is orthonormal. We can write the inner product of a and b in 
terms of their components in an orthonormal basis as 


(alb) = (a1@, + an@ + +++ + anOy|b1€; + b2@2 +--+ + byéy) 


N N WN 
= $7 a5 bi(Bilei) + >> D> abi (Bile) 
i=l i=l j#i 


where the second equality follows from (8.14) and the third from (8.15). This is 
clearly a generalisation of the expression (7.21) for the dot product of vectors in 
three-dimensional space. 

We may generalise the above to the case where the base vectors e1,€2,...,ey 
are not orthonormal (or orthogonal). In general we can define the N* numbers 


Gij => (ejle;). (8.17) 


Then, if a = pone aje; and b = eS bje;, the inner product of a and b is given by 


(alb) = (Sas Snes) 


N WN 
= Dd 4 Giphj. (8.18) 


We further note that from (8.17) and the properties of the inner product we 
require Gj; = Gj;. This in turn ensures that ||a|| = (ala) is real, since then 


N 
(alja)" ayy a Gjja; = 3 So aj Gjiai = (ala). 


i=1 j=l j=l i=1 
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8.1.3 Some useful inequalities 


For a set of objects (vectors) forming a linear vector space in which (ala) > 0 for 
all a, the following inequalities are often useful. 


(i) 


(ii 


har] 


(iii) 


Schwarz’s inequality is the most basic result and states that 
|(alb)| < |lal| bl), (8.19) 


where the equality holds when a is a scalar multiple of b, i.e. when a = Ab. 
It is important here to distinguish between the absolute value of a scalar, 
|A|, and the norm of a vector, ||a||. Schwarz’s inequality may be proved by 
considering 
ja + Abl|* = (a + Aba + Ab) 
= (ala) + A(alb) + 2° (bla) + 22° (bIb). 
If we write (a|b) as |(alb)|e” then 
Jja + Ab||? = llall? + [A |b? + AI (alb)|e* + 2° |(alb) |e. 


However, ||a + Ab||? > 0 for all A, so we may choose 2 = re~” and require 


that, for all r, 
0 < |la + Ab||? = |lal|? + r7||b||? + 2r|(a[b)|. 


This means that the quadratic equation in r formed by setting the RHS 
equal to zero must have no real roots. This, in turn, implies that 


4|(a|b)|? < 4|/a\||\b||*, 


which, on taking the square root (all factors are necessarily positive) of 
both sides, gives Schwarz’s inequality. 
The triangle inequality states that 


a + bl] < |lal| + |/b|| (8.20) 


and may be derived from the properties of the inner product and Schwarz’s 
inequality as follows. Let us first consider 


||a + b\|? = |lall? + |||? + 2 Re (alb) < jal]? + bl] + 2\(alb)|. 


Using Schwarz’s inequality we then have 
ja + Bll? < lal]? + |lbl]* + 2[lal|||bl| = (ilal] + bl), 


which, on taking the square root, gives the triangle inequality (8.20). 
Bessel’s inequality requires the introduction of an orthonormal basis 6, 
i= 1,2,...,N into the N-dimensional vector space; it states that 


lal? = So 1@ilay?, (8.21) 
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where the equality holds if the sum includes all N basis vectors. If not 
all the basis vectors are included in the sum then the inequality results 


(though of course the equality remains if those basis vectors omitted all 


have a; = 0). Bessel’s inequality can also be written 


(ala) > Dla 


where the a; are the components of a in the orthonormal basis. From (8.16) 
these are given by a; = (€;|a). The above may be proved by considering 


2 
a—)“(@la)éi] = 


i 


(a — DY Gilayéia — > (jla)é;). 


i Jj 


Expanding out the inner product and using (é|a)* = (alé;), we obtain 


2 


a— S>ila)éi] = 
i 


Now (@;|é;) = 6;;, since the basis is orthonormal, and so we find 


a—5_ila)éi| = llall? — DI ila)? 


i 


0< 


which is Bessel’s inequality. 
We take this opportunity to mention also 
(iv) the parallelogram equality 


l|a + bil? + la — bl|? = 2 (all? + bl”), 


(ala) — aos alé;) (é;|a) ee, a|é;) (€;|a) (€;/é;). 


(8.22) 


which may be proved straightforwardly from the properties of the inner 


product. 


8.2 Linear operators 


We now discuss the action of linear operators on vectors in a vector space. A 


linear operator A associates with every vector x another vector 


y= Ax, 


in such a way that, for two vectors a and b, 


A(ja + ub) = 2Aat+ pAD, 


where A, x are scalars. We say that A ‘operates’ on x to give the vector y. We 
note that the action of A is independent of any basis or coordinate system and 
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may be thought of as ‘transforming’ one geometrical entity (i.e. a vector) into 
another. 

If we now introduce a basis e;, i = 1,2,...,N, into our vector space then the 
action of A on each of the basis vectors is to produce a linear combination of 
the latter; this may be written as 


N 
Ae = >> Aiei, (8.23) 
i=1 


where Aj; is the ith component of the vector Ae; in this basis; collectively the 
numbers A;; are called the components of the linear operator in the ej-basis. In 
this basis we can express the relation y = Ax in component form as 


N N 


N N 
y=) wei =A | So xje)} => ox) >> Aven 
i=1 j=l jel i=l 


and hence, in purely component form, in this basis we have 
N 
w= >) Ayx;. (8.24) 
j=l 


If we had chosen a different basis ej, in which the components of x, y and A 
are x}, y; and Aj, respectively then the geometrical relationship y = Ax would be 
represented in this new basis by 


N 

TP ae 

Vie 5 Aj jx; 
j=l 


We have so far assumed that the vector y is in the same vector space as 
x. If, however, y belongs to a different vector space, which may in general be 
M-dimensional (M # N) then the above analysis needs a slight modification. By 
introducing a basis set f;, i = 1,2,..., M, into the vector space to which y belongs 
we may generalise (8.23) as 


M 
Ae; > S— Aili, 
i=1 


where the components Aj; of the linear operator A relate to both of the bases e; 
and f,;. 
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8.2.1 Properties of linear operators 
If x is a vector and A and B are two linear operators then it follows that 


(A +B)x=Ax+Bx, 
(A )x = (Ax), 
(AB )x = A(Bx), 
where in the last equality we see that the action of two linear operators in 
succession is associative. The product of two linear operators is not in general 


commutative, however, so that in general ABx + B Ax. In an obvious way we 
define the null (or zero) and identity operators by 


Ox=0 and Tx =x, 


for any vector x in our vector space. Two operators A and 6 are equal if 
Ax = Bx for all vectors x. Finally, if there exists an operator A~! such that 


AAS ATA ST 


then A! is the inverse of A. Some linear operators do not possess an inverse 
and are called singular, whilst those operators that do have an inverse are termed 
non-singular. 


8.3. Matrices 


We have seen that in a particular basis e; both vectors and linear operators 
can be described in terms of their components with respect to the basis. These 
components may be displayed as an array of numbers called a matrix. In general, 
if a linear operator A transforms vectors from an N-dimensional vector space, 


for which we choose a basis e;, j = 1,2,...,N, into vectors belonging to an 
M-dimensional vector space, with basis f;, i = 1,2,...,M, then we may represent 
the operator A by the matrix 
Ai Aj? eee Ain 
Ax An see Ain 
A= ; ; : (8.25) 
Ami Am2 ... Amn 


The matrix elements Aj; are the components of the linear operator with respect 
to the bases e; and fj; the component Aj; of the linear operator appears in the 
ith row and jth column of the matrix. The array has M rows and N columns 
and is thus called an M x N matrix. If the dimensions of the two vector spaces 
are the same, ic. M = N (for example, if they are the same vector space) then we 
may represent A by an N x N or square matrix of order N. The component Ajj, 
which in general may be complex, is also denoted by (A)jj. 
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In a similar way we may denote a vector x in terms of its components x; in a 
basis e;, i = 1,2,...,.N, by the array 


which is a special case of (8.25) and is called a column matrix (or conventionally, 
and slightly confusingly, a column vector or even just a vector — strictly speaking 
the term ‘vector’ refers to the geometrical entity x). The column matrix x can also 
be written as 


T 
X=(X1 X2 +++ XN), 


which is the transpose of a row matrix (see section 8.6). 
We note that in a different basis e; the vector x would be represented by a 
different column matrix containing the components x; in the new basis, i.e. 
x 
X4 


xy 
Thus, we use x and x’ to denote different column matrices which, in different bases 
e; and ej, represent the same vector x. In many texts, however, this distinction is 
not made and x (rather than x) is equated to the corresponding column matrix; if 
we regard x as the geometrical entity, however, this can be misleading and so we 
explicitly make the distinction. A similar argument follows for linear operators; 
the same linear operator A is described in different bases by different matrices A 
and A’, containing different matrix elements. 


8.4 Basic matrix algebra 


The basic algebra of matrices may be deduced from the properties of the linear 
operators that they represent. In a given basis the action of two linear operators 
A and B on an arbitrary vector x (see the beginning of subsection 8.2.1), when 
written in terms of components using (8.24), is given by 


So(A + B)iixy = D> Aix; + D5 Bij, 
j j i 
YEO) iX) = 4 Ai? 
j Jj 
S (AB) jx; = So Ain(Bx)x => [> S > An Bujxj- 
j k jk 
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Now, since x is arbitrary, we can immediately deduce the way in which matrices 
are added or multiplied, ie. 


(AA)ij = AAij, (8.27) 
(AB)ij = 55 Aix Bry. (8.28) 
k 


We note that a matrix element may, in general, be complex. We now discuss 
matrix addition and multiplication in more detail. 


8.4.1 Matrix addition and multiplication by a scalar 


From (8.26) we see that the sum of two matrices, S = A+B, is the matrix whose 
elements are given by 


Sij = Aij + Bij 


for every pair of subscripts i,j, with i = 1,2,...,.M and j = 1,2,...,N. For 
example, if A and B are 2 x 3 matrices then S = A+ B is given by 


( Si Sia S43 ) 7 ( An Ap A )+( By By By ) 
Sop Say S93 Ay, An Ax3 By, By By 
- ( Ant+By Aint By Ap t+ By3 ) 
Ax, + Bo A227 + Br. Ar3 + Bo3 ] 


(8.29) 


Clearly, for the sum of two matrices to have any meaning, the matrices must have 
the same dimensions, i.e. both be M x N matrices. 

From definition (8.29) it follows that A+ B = B+A and that the sum of a 
number of matrices can be written unambiguously without bracketting, i.e. matrix 
addition is commutative and associative. 

The difference of two matrices is defined by direct analogy with addition. The 
matrix D = A—B has elements 


Dj; = Ajj — Bij, fori =1,2,...,M, j =1,2,...,N. (8.30) 


From (8.27) the product of a matrix A with a scalar 4 is the matrix with 
elements /Aj;, for example 


ZO A12 a ein AAr2 ) (8.31) 
Ay, Az. Ar3 2An 2Ax 2A ]* ‘ 


Multiplication by a scalar is distributive and associative. 
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> The matrices A, B and C are given by 


e(F7) e2(6 4). o=(2 4): 


Find the matrix D=A+2B—C. 


o=(3 1 )+2(0 %)-(2 1) 


_ ( 24+2x1-(-2) -14+2x0-1 \_/(6 -2 
Th 4S 01) La ead YT cade J 


From the above considerations we see that the set of all, in general complex, 
M x N matrices (with fixed M and N) forms a linear vector space of dimension 
MN. One basis for the space is the set of M x N matrices E'’”) with the property 
that EG?) = 1 ifi=p and j =q whilst ae = 0 for all other values of i and 
j, Le. each matrix has only one non-zero entry, which equals unity. Here the pair 
(p,q) is simply a label that picks out a particular one of the matrices E?”), the 
total number of which is MN. 


8.4.2 Multiplication of matrices 


Let us consider again the ‘transformation’ of one vector into another, y = Ax, 
which, from (8.24), may be described in terms of components with respect to a 
particular basis as 


N 
yi = So Ayx; for i=1,2,...,M. (8.32) 
j=l 


Writing this in matrix form as y = Ax we have 


V1 Ay Aj... Ain 


2 
eee (8.33) 


YM Ami Ayzr .-- Amn 


where we have highlighted with boxes the components used to calculate the 
element y2: using (8.32) for i = 2, 


V2 = Agyx1 + Arx2 + +++ + Arn. 


All the other components y; are calculated similarly. 
If instead we operate with A on a basis vector e; having all components zero 
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except for the jth, which equals unity, then we find 


Ay Ar Ain Ajj 
Ay, Any Aon Adj 

Ae; = ‘ == 2 > 
Amit Am? Amn Amj 


and so confirm our identification of the matrix element Aj; as the ith component 
of Ae; in this basis. 

From (8.28) we can extend our discussion to the product of two matrices 
P = AB, where P is the matrix of the quantities formed by the operation of 
the rows of A on the columns of B, treating each column of B in turn as the 
vector x represented in component form in (8.32). It is clear that, for this to be 
a meaningful definition, the number of columns in A must equal the number of 
rows in B. Thus the product AB of an M x N matrix A with an N x R matrix B 
is itself an M x R matrix P, where 


N 
Py = So Ain Bu fori=1,2,...,.M, j=1,2,...,R. 
k=1 


For example, P = AB may be written in matrix form 


B B 
( Pip =( Ay Ap Aj3 ) a ee 


Px Px An, Az. Ar3 


where 


Py = Ay By + A12Bar + A13Bai, 
Px, = Az By + Ax Bar + Ar3Ba1, 
Pip = Ay By + Ar.Bx + A13 B32, 
Py = Api By + Ax Br + A23B32. 


Multiplication of more than two matrices follows naturally and is associative. 
So, for example, 


A(BC) = (AB)C, (8.34) 


provided, of course, that all the products are defined. 
As mentioned above, if A is an M x N matrix and B is an N x M matrix then 
two product matrices are possible, i.e. 


P=AB and Q=BA. 
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These are clearly not the same, since P is an M x M matrix whilst Q is an 
N x N matrix. Thus, particular care must be taken to write matrix products in 
the intended order; P = AB but Q = BA. We note in passing that A? means AA, 
A> means A(AA) = (AA)A etc. Even if both A and B are square, in general 


AB BA, (8.35) 


ie. the multiplication of matrices is not, in general, commutative. 


> Evaluate P = AB and Q= BA where 


As we saw for the 2 x 2 case above, the element Pj; of the matrix P = AB is found by 
mentally taking the ‘scalar product’ of the ith row of A with the jth column of B. For 
example, Pj) =3 x 2+2x1+(-1)x3=5, Py =3 x (—-2)+2x 1+ (-1) x2 =—6, ete. 


Thus 
35-5 2°  =1 
P=AB={ 0 3 2 
1 -3 4 


and, similarly, 


2 —2 3 3 2 -1 9 -l11 6 
Q=BA=; 1 1 O 0 3 2 = 3 B} He {Jig 
clans ane | 1 -—3 4 10 9 5 


These results illustrate that, in general, two matrices do not commute. < 


The property that matrix multiplication is distributive over addition, i.e. that 
(A+ B)C = AC+ BC (8.36) 
and 
C(A +B) = CA+CB, (8.37) 


follows directly from its definition. 


8.4.3 The null and identity matrices 


Both the null matrix and the identity matrix are frequently encountered, and we 
take this opportunity to introduce them briefly, leaving their uses until later. The 
null or zero matrix 0 has all elements equal to zero, and so its properties are 


AO =0=0A, 
A+0=0+A=A. 
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The identity matrix | has the property 
Al=IA=A. 


It is clear that, in order for the above products to be defined, the identity matrix 
must be square. The N x N identity matrix (often denoted by Iv) has the form 


1 0 0 
0 1 
ly = 
ae 
0 0 1 


8.5 Functions of matrices 


If a matrix A is square then, as mentioned above, one can define powers of A in 
a straightforward way. For example A? = AA, A? = AAA, or in the general case 


A" =AA::-A (n times), 


where n is a positive integer. Having defined powers of a square matrix A, we 
may construct functions of A of the form 


S= Sanh”, 
n 


where the a, are simple scalars and the number of terms in the summation may 
be finite or infinite. In the case where the sum has an infinite number of terms, 
the sum has meaning only if it converges. A common example of such a function 
is the exponential of a matrix, which is defined by 


expA=So—. (8.38) 


This definition can, in turn, be used to define other functions such as sinA and 
cosA. 


8.6 The transpose of a matrix 


We have seen that the components of a linear operator in a given coordinate sys- 
tem can be written in the form of a matrix A. We will also find it useful, however, 
to consider the different (but clearly related) matrix formed by interchanging the 
rows and columns of A. The matrix is called the transpose of A and is denoted 
by AT. 
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> Find the transpose of the matrix 


By interchanging the rows and columns of A we immediately obtain 
3 0 
AT={] 1 4 |. 
2 1 


It is obvious that if A is an M x N matrix then its transpose A? is a N x M 
matrix. As mentioned in section 8.3, the transpose of a column matrix is a 
row matrix and vice versa. An important use of column and row matrices is 
in the representation of the inner product of two real vectors in terms of their 
components in a given basis. This notion is discussed fully in the next section, 
where it is extended to complex vectors. 

The transpose of the product of two matrices, (AB)', is given by the product 
of their transposes taken in the reverse order, i.e. 


(AB)' = BTA’. (8.39) 
This is proved as follows: 
(AB)j; = (AB) = S > Aj Bui 
k 


= SoA i(B ik = SOB KA Ki = (BIA')i, 
k 


k 


and the proof can be extended to the product of several matrices to give 


(ABC---G)' =G!..-cTBTAT. 


8.7 The complex and Hermitian conjugates of a matrix 


Two further matrices that can be derived from a given general M x N matrix 
are the complex conjugate, denoted by A“, and the Hermitian conjugate, denoted 
by At. 
The complex conjugate of a matrix A is the matrix obtained by taking the 
complex conjugate of each of the elements of A, i.e. 
(A")ij = (Ay). 


Obviously if a matrix is real (i.e. it contains only real elements) then A* = A. 
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> Find the complex conjugate of the matrix 


ees 
a=( ah, 1 wae 


By taking the complex conjugate of each element we obtain immediately 
Sh SAN Pst 
in ( ieg  G ) a 
The Hermitian conjugate, or adjoint, of a matrix A is the transpose of its 
complex conjugate, or equivalently, the complex conjugate of its transpose, i.e. 
At =(A‘)’ =(A‘)’. 


We note that if A is real (and so A* = A) then At = A’, and taking the Hermitian 
conjugate is equivalent to taking the transpose. Following the previous line of 
argument for the transpose of the product of several matrices, the Hermitian 
conjugate of such a product can be shown to be given by 


(AB---G)i = Gl... Bray, (8.40) 


> Find the Hermitian conjugate of the matrix 


ese oH 
a=( 4h, 1 ee 


Taking the complex conjugate of A and then forming the transpose we find 


1 
At= 2 1 
—3i (0) 
We obtain the same result, of course, if we first take the transpose of A and then take the 
complex conjugate. < 


An important use of the Hermitian conjugate (or transpose in the real case) 
is in connection with the inner product of two vectors. Suppose that in a given 
orthonormal basis the vectors a and b may be represented by the column matrices 


ay by 
ay bo 

a= ; and b= : : (8.41) 
an by 


Taking the Hermitian conjugate of a, to give a row matrix, and multiplying (on 
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the right) by b we obtain 


by 
by ui 
alb=(aja}---ay)] . | = do ajbi, (8.42) 
: i=1 
by 


which is the expression for the inner product (a|b) in that basis. We note that for 
real vectors (8.42) reduces to ab = 7), abi. 
If the basis e; is not orthonormal, so that, in general, 


(eie;) = Gi; # di, 
then, from (8.18), the scalar product of a and b in terms of their components with 


respect to this basis is given by 


N WN 


(alb) = +S » aj Gijbj = al Gb, 


i=1 j=l 


where G is the N x N matrix with elements Gj;. 


8.8 The trace of a matrix 


For a given matrix A, in the previous two sections we have considered various 
other matrices that can be derived from it. However, sometimes one wishes to 
derive a single number from a matrix. The simplest example is the trace (or spur) 
of a square matrix, which is denoted by TrA. This quantity is defined as the sum 
of the diagonal elements of the matrix, 


N 
TrA = Ay + An +--+ + Ann = >) Ai. (8.43) 
i=1 


It is clear that taking the trace is a linear operation so that, for example, 
Tr(A+B)=TrA+TrB. 


A very useful property of traces is that the trace of the product of two matrices 
is independent of the order of their multiplication; this results holds whether or 
not the matrices commute and is proved as follows: 


N N WN N N 


N 


i=1 i=1 j=l i=1 j=l j=l (8.44) 


The result can be extended to the product of several matrices. For example, from 
(8.44), we immediately find 


Tr ABC = Tr BCA = Tr CAB, 
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which shows that the trace of a multiple product is invariant under cyclic 
permutations of the matrices in the product. Other easily derived properties of 
the trace are, for example, Tr A’ = Tr A and Tr At = (Tr A)’. 


8.9 The determinant of a matrix 


For a given matrix A, the determinant det A (like the trace) is a single number (or 
algebraic expression) that depends upon the elements of A. Also like the trace, 
the determinant is defined only for square matrices. If, for example, A is a 3 x 3 
matrix then its determinant, of order 3, is denoted by 


An Ap A 
det A= |A| => An, An Ax : (8.45) 
Az, A32 A33 


In order to calculate the value of a determinant, we first need to introduce 
the notions of the minor and the cofactor of an element of a matrix. (We 
shall see that we can use the cofactors to write an order-3 determinant as the 
weighted sum of three order-2 determinants, thereby simplifying its evaluation.) 
The minor M;; of the element A;; of an N x N matrix A is the determinant of 
the (N — 1) x (N — 1) matrix obtained by removing all the elements of the ith 
row and jth column of A; the associated cofactor, C;;, is found by multiplying 
the minor by (—1)*/, 


> Find the cofactor of the element A>; of the matrix 


Au Aw Ap 
A={ An An Adz 
Az1 Az. A33 


Removing all the elements of the second row and third column of A and forming the 
determinant of the remaining terms gives the minor 


An A 


oe | Ax, A32 


Multiplying the minor by (—1)?+3 = (—1)> = —1 gives 


We now define a determinant as the sum of the products of the elements of any 
row or column and their corresponding cofactors, ¢.g. A2,C2, + A22C2 + A23C23 or 
Ay3C13 + A23C23 + A33C33. Such a sum is called a Laplace expansion. For example, 
in the first of these expansions, using the elements of the second row of the 
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determinant defined by (8.45) and their corresponding cofactors, we write |A| as 
the Laplace expansion 


JA] = Aai(—1)°F? Moy + Ago(—1)P*) Moy + Aa3(—1)°*?) Mp3 
Ay. Aj3 Ay A | - | Ay, Ap 
Az, Ax 


We will see later that the value of the determinant is independent of the row 
or column chosen. Of course, we have not yet determined the value of |A| but, 
rather, written it as the weighted sum of three determinants of order 2. However, 
applying again the definition of a determinant, we can evaluate each of the 
order-2 determinants. 


> Evaluate the determinant 


Ay Ais 
A3r A33 


By considering the products of the elements of the first row in the determinant, and their 
corresponding cofactors, we find 
Ay A 
42 48 f= Ani an| + Anya 


= Aj2A33 — A13A30, 


where the values of the order-1 determinants |A33| and |A32| are defined to be A33 and A32 
respectively. It must be remembered that the determinant is not the same as the modulus, 
e.g. det (—2) = | — 2| = —2, not 2. < 


We can now combine all the above results to show that the value of the 
determinant (8.45) is given by 


|A| = —A21(A12A33 — A13A32) + Ar2(A11.433 — A13A31) 


— Ao3(A11A32 — A12A31) (8.46) 
= Aj1(A22A33 — A23A32) + A12(A23431 — Ari A33) 
+ A13(A21A32 — A22A31), (8.47) 


where the final expression gives the form in which the determinant is usually 
remembered and is the form that is obtained immediately by considering the 
Laplace expansion using the first row of the determinant. The last equality, which 
essentially rearranges a Laplace expansion using the second row into one using 
the first row, supports our assertion that the value of the determinant is unaffected 
by which row or column is chosen for the expansion. 
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> Suppose the rows of a real 3 x 3 matrix A are interpreted as the components in a given 
basis of three (three-component ) vectors a, b and ec. Show that one can write the determinant 
of A as 


|A| =a- (bx ce). 


If one writes the rows of A as the components in a given basis of three vectors a, b and c, 
we have from (8.47) that 


a4 a2 a3 
JA} =| b1 by b3_ | = ay(by03 — b3cz) + a2(b3¢1 — b1c3) + a3(b1c2 — bae1). 


Cr C2 C3 


From expression (7.34) for the scalar triple product given in subsection 7.6.3, it follows 
that we may write the determinant as 


|A| =a- (bx). (8.48) 


In other words, |A| is the volume of the parallelepiped defined by the vectors a, b and 
c. (One could equally well interpret the columns of the matrix A as the components of 
three vectors, and result (8.48) would still hold.) This result provides a more memorable 
(and more meaningful) expression than (8.47) for the value of a 3 x 3 determinant. Indeed, 
using this geometrical interpretation, we see immediately that, if the vectors a;, a2, a3 are 
not linearly independent then the value of the determinant vanishes: |A| = 0. < 


The evaluation of determinants of order greater than 3 follows the same general 
method as that presented above, in that it relies on successively reducing the order 
of the determinant by writing it as a Laplace expansion. Thus, a determinant 
of order 4 is first written as a sum of four determinants of order 3, which 
are then evaluated using the above method. For higher-order determinants, one 
cannot write down directly a simple geometrical expression for |A| analogous to 
that given in (8.48). Nevertheless, it is still true that if the rows or columns of 
the N x N matrix A are interpreted as the components in a given basis of N 
(N-component) vectors a;,a,...,a,, then the determinant |A| vanishes if these 
vectors are not all linearly independent. 


8.9.1 Properties of determinants 


A number of properties of determinants follow straightforwardly from the defini- 
tion of det A; their use will often reduce the labour of evaluating a determinant. 
We present them here without specific proofs, though they all follow readily from 
the alternative form for a determinant, given in equation (26.29) on page 942, 
and expressed in terms of the Levi—Civita symbol €;; (see exercise 26.9). 


(i) Determinant of the transpose. The transpose matrix A’ (which, we recall, 
is obtained by interchanging the rows and columns of A) has the same 
determinant as A itself, i.e. 


|AT| =|Al. (8.49) 
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(ii) 


(iii) 


(iv) 


(v) 


(vi) 


(vii) 


It follows that any theorem established for the rows of A will apply to the 
columns as well, and vice versa. 

Determinant of the complex and Hermitian conjugate. It is clear that the 
matrix A* obtained by taking the complex conjugate of each element of A 
has the determinant |A*| = |A|*. Combining this result with (8.49), we find 
that 


JAY] = |(A")"] = [A] = IAN". (8.50) 


Interchanging two rows or two columns. If two rows (columns) of A are 
interchanged, its determinant changes sign but is unaltered in magnitude. 
Removing factors. If all the elements of a single row (column) of A have 
a common factor, 4, then this factor may be removed; the value of the 
determinant is given by the product of the remaining determinant and /. 
Clearly this implies that if all the elements of any row (column) are zero 
then |A| = 0. It also follows that if every element of the N x N matrix A 
is multiplied by a constant factor 2 then 


|AA| = AAI. (8.51) 


Identical rows or columns. If any two rows (columns) of A are identical or 
are multiples of one another, then it can be shown that |A| = 0. 


Adding a constant multiple of one row (column ) to another. The determinant 
of a matrix is unchanged in value by adding to the elements of one row 
(column) any fixed multiple of the elements of another row (column). 


Determinant of a product. If A and B are square matrices of the same order 
then 


|AB| = |A||B| = |BAI. (8.52) 
A simple extension of this property gives, for example, 
|AB---G| = |A||B]--- |G) = |A||G| --- |B] = |A--- GBI, 


which shows that the determinant is invariant under permutation of the 
matrices in a multiple product. 


There is no explicit procedure for using the above results in the evaluation of 


any given determinant, and judging the quickest route to an answer is a matter 


of experience. A general guide is to try to reduce all terms but one in a row or 


column to zero and hence in effect to obtain a determinant of smaller size. The 


steps taken in evaluating the determinant in the example below are certainly not 
the fastest, but they have been chosen in order to illustrate the use of most of the 
properties listed above. 
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> Evaluate the determinant 


in Ga eee 3 
Oe enor a 
labret g a 


Taking a factor 2 out of the third column and then adding the second column to the third 
gives 


1 0 1 3 1 0 1 3 
W=2) 3 43 > »/=2)3 4 4 4 
24 1 1 25 ul 0 -1 
Subtracting the second column from the fourth gives 
1 0 1 3 
W=2) 5 43 3 4 


—2 #1 0 -2 


We now note that the second row has only one non-zero element and so the determinant 
may conveniently be written as a Laplace expansion, Le. 


1 1 3 4 0 4 


|A|=2x1x(-1P"?} 3 -1 1 |=2]/ 3° -1 1 |, 
2 0 2 2 0 2 


where the last equality follows by adding the second row to the first. It can now be seen 
that the first row is minus twice the third, and so the value of the determinant is zero, by 
property (v) above. < 


8.10 The inverse of a matrix 


Our first use of determinants will be in defining the inverse of a matrix. If we 
were dealing with ordinary numbers we would consider the relation P = AB as 
equivalent to B = P/A, provided that A # 0. However, if A, B and P are matrices 
then this notation does not have an obvious meaning. What we really want to 
know is whether an explicit formula for B can be obtained in terms of A and 
P. It will be shown that this is possible for those cases in which |A| # 0. A 
square matrix whose determinant is zero is called a singular matrix; otherwise it 
is non-singular. We will show that if A is non-singular we can define a matrix, 
denoted by A~! and called the inverse of A, which has the property that if AB = P 
then B = A“'P. In words, B can be obtained by multiplying P from the left by 
A~!. Analogously, if B is non-singular then, by multiplication from the right, 
A=PB!. 
It is clear that 


AIS Ao hes. CSA, (8.53) 


where | is the unit matrix, and so A7'A = | = AA7!. These statements are 
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equivalent to saying that if we first multiply a matrix, B say, by A and then 
multiply by the inverse A~!, we end up with the matrix we started with, ie. 


A AB =B. (8.54) 


This justifies our use of the term inverse. It is also clear that the inverse is only 
defined for square matrices. 

So far we have only defined what we mean by the inverse of a matrix. Actually 
finding the inverse of a matrix A may be carried out in a number of ways. We will 
show that one method is to construct first the matrix C containing the cofactors 
of the elements of A, as discussed in the last subsection. Then the required inverse 
A~' can be found by forming the transpose of C and dividing by the determinant 
of A. Thus the elements of the inverse A~! are given by 

1 (Chik = Cri 
(Aik = Al Al" (8.55) 

That this procedure does indeed result in the inverse may be seen by considering 

the components of A~'A, ice. 


C 7 A 
(ATA) = So(a- Yea = » atti -mo ij. (8.56) 
k 


The last equality in (8.56) relies on the property 


2, Cid = |Aloij; (8.57) 


this can be proved by considering the matrix A’ obtained from the original matrix 
A when the ith column of A is replaced by one of the other columns, say the jth. 
Thus A’ is a matrix with two identical columns and so has zero determinant. 
However, replacing the ith column by another does not change the cofactors Cx; 
of the elements in the ith column, which are therefore the same in A and A’. 
Recalling the Laplace expansion of a determinant, ie. 


IAI = So AgiCri, 
k 


we obtain 


O=|A|=)> AUC => Ayla, FF 5, 
k k 


which together with the Laplace expansion itself may be summarised by (8.57). 
It is immediately obvious from (8.55) that the inverse of a matrix is not defined 
if the matrix is singular (ie. if |A] = 0). 
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> Find the inverse of the matrix 


We first determine |A|: 


|A| = 2[—2(2) — (—2)3] + 4[(—2)(—3) — (1)(2)] + 310)(3) — (—2)(-3)] 
=11. (8.58) 


This is non-zero and so an inverse matrix can be constructed. To do this we need the 
matrix of the cofactors, C, and hence C!. We find 


2 4 -3 2 1 —2 
C= 1 13 —18 and Cl= 4 13 7 ; 
—2 7 -8 —3 -18 -8 


1 a 
mana | 13 i) (8.59) 


and hence 


For a 2 x 2 matrix, the inverse has a particularly simple form. If the matrix is 


Ay At ) 
A= 
( An, Ary 


then its determinant |A| is given by |A] = 411422 — Ai2A21, and the matrix of 


cofactors is 
Ay —Adi ) 
C= F 
( —-Ay Ai 
Thus the inverse of A is given by 


ct 1 ( Ay —Ajy ) 
At = — = — : 8.60 
|A| = AtjA22 — Ay2An1 \ —Arr = Att ee) 


It can be seen that the transposed matrix of cofactors for a 2 x 2 matrix is the 
same as the matrix formed by swapping the elements on the leading diagonal 
(Ay, and A») and changing the signs of the other two elements (Aj and A);). 
This is completely general for a 2 x 2 matrix and is easy to remember. 

The following are some further useful properties related to the inverse matrix 
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and may be straightforwardly derived. 


(i) (ATS TSA. 
(ii) (AT) 1 = (AT). 
(iii) (AT)“! = (A7)t. 
(iv) (AB)-! = BAT, 
(v) (AB---G)!=G!..-BtAt 


> Prove the properties (i)-(v) stated above. 


We begin by writing down the fundamental expression defining the inverse of a non- 
singular square matrix A: 
AAT =1=ATA. (8.61) 
Property (i). This follows immediately from the expression (8.61). 
Property (ii). Taking the transpose of each expression in (8.61) gives 
(AA')7 = 17 = (ATA)’. 
Using the result (8.39) for the transpose of a product of matrices and noting that I’ = |, 
we find 
(A)TAT == AN(A)!, 
However, from (8.61), this implies (A~!)' = (AT)! and hence proves result (ii) above. 
Property (iii). This may be proved in an analogous way to property (ii), by replacing the 
transposes in (ii) by Hermitian conjugates and using the result (8.40) for the Hermitian 


conjugate of a product of matrices. 
Property (iv). Using (8.61), we may write 


(AB)(AB)' = | = (AB)"'(AB), 
From the left-hand equality it follows, by multiplying on the left by A~', that 
A~'AB(AB)“! = A“1I and hence B(AB)~! = AM. 


Now multiplying on the left by B~! gives 
B-'B(AB)"! = BIA-1, 
and hence the stated result. 


Property (v). Finally, result (iv) may extended to case (v) in a straightforward manner. 
For example, using result (iv) twice we find 


(ABC)-! = (BC)"'A! = CBA“! 


We conclude this section by noting that the determinant |A~!| of the inverse 
matrix can be expressed very simply in terms of the determinant |A| of the matrix 
itself. Again we start with the fundamental expression (8.61). Then, using the 
property (8.52) for the determinant of a product, we find 


|AA~!| = |AI|A7!| = I. 


It is straightforward to show by Laplace expansion that |I| = 1, and so we arrive 
at the useful result 
1 


JA} = —. (8.62) 
Al 
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8.11 The rank of a matrix 


The rank of a general M x N matrix is an important concept, particularly in 
the solution of sets of simultaneous linear equations, to be discussed in the next 
section, and we now discuss it in some detail. Like the trace and determinant, 
the rank of matrix A is a single number (or algebraic expression) that depends 
on the elements of A. Unlike the trace and determinant, however, the rank of a 
matrix can be defined even when A is not square. As we shall see, there are two 
equivalent definitions of the rank of a general matrix. 

Firstly, the rank of a matrix may be defined in terms of the linear independence 
of vectors. Suppose that the columns of an M x N matrix are interpreted as 


the components in a given basis of N (M-component) vectors v1,V2,...,VN, as 
follows: 
tT T ii 
A= Vi V2 VN 
L J 1 
Then the rank of A, denoted by rank A or by R(A), is defined as the number 
of linearly independent vectors in the set v1,V2,...,Vvv, and equals the dimension 


of the vector space spanned by those vectors. Alternatively, we may consider the 
rows of A to contain the components in a given basis of the M (N-component) 
vectors W1,W2,...,Wy as follows: 


<-— Wu > 


It may then be shown’ that the rank of A is also equal to the number of 
linearly independent vectors in the set w,,W2,...,Wy. From this definition it is 
should be clear that the rank of A is unaffected by the exchange of two rows 
(or two columns) or by the multiplication of a row (or column) by a constant. 
Furthermore, suppose that a constant multiple of one row (column) is added to 
another row (column): for example, we might replace the row w; by w; + cwj. 
This also has no effect on the number of linearly independent rows and so leaves 
the rank of A unchanged. We may use these properties to evaluate the rank of a 
given matrix. 

A second (equivalent) definition of the rank of a matrix may be given and uses 
the concept of submatrices. A submatrix of A is any matrix that can be formed 
from the elements of A by ignoring one, or more than one, row or column. It 


S For a fuller discussion, see, for example, C. D. Cantrell, Modern Mathematical Methods for Physicists 
and Engineers (Cambridge: Cambridge University Press, 2000), chapter 6. 
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may be shown that the rank of a general M x N matrix is equal to the size of 
the largest square submatrix of A whose determinant is non-zero. Therefore, if a 
matrix A has anr xr submatrix S with |S| 4 0, but no (r+ 1) x (r+ 1) submatrix 
with non-zero determinant then the rank of the matrix is r. From either definition 
it is clear that the rank of A is less than or equal to the smaller of M and N. 


> Determine the rank of the matrix 


1 10 -—2 
A=|| 2 0 2 2 
4 els 3 il 


The largest possible square submatrices of A must be of dimension 3 x 3. Clearly, A 
possesses four such submatrices, the determinants of which are given by 


1 1 0 1 1 —2 
2 0 2/=0, 2 0 2 |=0, 
4 1 3 4 1 1 
1 0 -—2 1 0 -—2 
2. 2. 2. | 0; 0 2 2 |=0. 
4 3. 1 13 A 


(In each case the determinant may be evaluated as described in subsection 8.9.1.) 

The next largest square submatrices of A are of dimension 2 x 2. Consider, for example, 
the 2 x 2 submatrix formed by ignoring the third row and the third and fourth columns 
of A; this has determinant 


1 1 


2 0 |=1x0-2%1=-2 


Since its determinant is non-zero, A is of rank 2 and we need not consider any other 2 x 2 
submatrix. < 


In the special case in which the matrix A is a square N x N matrix, by comparing 
either of the above definitions of rank with our discussion of determinants in 
section 8.9, we see that |A| = 0 unless the rank of A is N. In other words, A is 
singular unless R(A) = N. 


8.12 Special types of square matrix 


Matrices that are square, i.e. N x N, are very common in physical applications. 
We now consider some special forms of square matrix that are of particular 
importance. 


8.12.1 Diagonal matrices 


The unit matrix, which we have already encountered, is an example of a diagonal 
matrix. Such matrices are characterised by having non-zero elements only on the 
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leading diagonal, i.e. only elements Aj; with i= j may be non-zero. For example, 


10 0 
A={0 2 0 : 
0 0 —3 


is a 3 x 3 diagonal matrix. Such a matrix is often denoted by A = diag (1, 2, —3). 
By performing a Laplace expansion, it is easily shown that the determinant of an 
N x N diagonal matrix is equal to the product of the diagonal elements. Thus, if 
the matrix has the form A = diag(A11, Az,..., Ann) then 


|A| = A11A22--- Ann. (8.63) 


Moreover, it is also straightforward to show that the inverse of A is also a 
diagonal matrix given by 


1 1 1 
A! =di ere . 
ie Car aa) 


Finally, we note that, if two matrices A and B are both diagonal then they have 
the useful property that their product is commutative: 


AB = BA. 


This is not true for matrices in general. 


8.12.2 Lower and upper triangular matrices 


A square matrix A is called lower triangular if all the elements above the principal 
diagonal are zero. For example, the general form for a 3 x 3 lower triangular 
matrix is 


Ag 60 30 
A=| An An 0 , 
Az, A327 A33 


where the elements Aj; may be zero or non-zero. Similarly an upper triangular 
square matrix is one for which all the elements below the principal diagonal are 
zero. The general 3 x 3 form is thus 


Ay Ay A 
0. © Ax 


By performing a Laplace expansion, it is straightforward to show that, in the 
general N x N case, the determinant of an upper or lower triangular matrix is 
equal to the product of its diagonal elements, 


|A| = Aq A220 -+- Ann. (8.64) 
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Clearly result (8.63) for diagonal matrices is a special case of this result. Moreover, 
it may be shown that the inverse of a non-singular lower (upper) triangular matrix 
is also lower (upper) triangular. 


8.12.3 Symmetric and antisymmetric matrices 


A square matrix A of order N with the property A = A? is said to be symmetric. 
Similarly a matrix for which A = —A! is said to be anti- or skew-symmetric 
and its diagonal elements a1, d22,...,ayy are necessarily zero. Moreover, if A is 
(anti-)symmetric then so too is its inverse A~!. This is easily proved by noting 
that if A= +A! then 


(AHP = (ATT = tA. 
Any N x N matrix A can be written as the sum of a symmetric and an 
antisymmetric matrix, since we may write 
A=4(A+A')+43(A—A')=B+C, 


where clearly B = B! and C = —C!. The matrix B is therefore called the 
symmetric part of A, and C is the antisymmetric part. 


>If A is an N x N antisymmetric matrix, show that |A| = 0 if N is odd. 


If A is antisymmetric then A’ = —A. Using the properties of determinants (8.49) and 
(8.51), we have 


|A| = |AT| =|— A] = (-1)A\. 
Thus, if N is odd then |A| = —|A|, which implies that |A| = 0. < 


8.12.4 Orthogonal matrices 
A non-singular matrix with the property that its transpose is also its inverse, 
AT=A71, (8.65) 


is called an orthogonal matrix. It follows immediately that the inverse of an 
orthogonal matrix is also orthogonal, since 


(A7!)? = (AT)! = (A!) 


Moreover, since for an orthogonal matrix ATA = I, we have 


|ATA| = |AT|A] = IA)? = II = 1. 


Thus the determinant of an orthogonal matrix must be |A] = +1. 
An orthogonal matrix represents, in a particular basis, a linear operator that 
leaves the norms (lengths) of real vectors unchanged, as we will now show. 
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Suppose that y = Ax is represented in some coordinate system by the matrix 
equation y = Ax; then (yly) is given in this coordinate system by 
yly = xTATAx = x!x. 


Hence (yly) = (x|x), showing that the action of a linear operator represented by 
an orthogonal matrix does not change the norm of a real vector. 


8.12.5 Hermitian and anti-Hermitian matrices 


An Hermitian matrix is one that satisfies A = A‘, where At is the Hermitian conju- 
gate discussed in section 8.7. Similarly if A‘ = —A, then A is called anti-Hermitian. 
A real (anti-)symmetric matrix is a special case of an (anti-)Hermitian matrix, in 
which all the elements of the matrix are real. Also, if A is an (anti-)Hermitian 
matrix then so too is its inverse A~', since 


(Ati =(Aty! =+at 


Any N x N matrix A can be written as the sum of an Hermitian matrix and 
an anti-Hermitian matrix, since 


A=5(A+A‘) + 4(A—AlT) =B+C, 


where clearly B = Bt and C = —C?. The matrix B is called the Hermitian part of 
A, and C is called the anti-Hermitian part. 


8.12.6 Unitary matrices 
A unitary matrix A is defined as one for which 
At=ATt, (8.66) 


Clearly, if A is real then At = A!, showing that a real orthogonal matrix is a 
special case of a unitary matrix, one in which all the elements are real. We note 
that the inverse A~! of a unitary is also unitary, since 


(AHP = (AT = (AT 
Moreover, since for a unitary matrix A'A = |, we have 


|A'A| = |A‘||A| = |Al|A] = || = 1. 


Thus the determinant of a unitary matrix has unit modulus. 

A unitary matrix represents, in a particular basis, a linear operator that leaves 
the norms (lengths) of complex vectors unchanged. If y = Ax is represented in 
some coordinate system by the matrix equation y = Ax then (yly) is given in this 
coordinate system by 


yty = xt ATAx = xix. 
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Hence (yl/y) = (x|x), showing that the action of the linear operator represented by 
a unitary matrix does not change the norm of a complex vector. The action of a 
unitary matrix on a complex column matrix thus parallels that of an orthogonal 
matrix acting on a real column matrix. 


8.12.7 Normal matrices 
A final important set of special matrices consists of the normal matrices, for which 
AA' = ATA, 


i.e. a normal matrix is one that commutes with its Hermitian conjugate. 

We can easily show that Hermitian matrices and unitary matrices (or symmetric 
matrices and orthogonal matrices in the real case) are examples of normal 
matrices. For an Hermitian matrix, A= A‘ and so 


AAT = AA= ATA, 
Similarly, for a unitary matrix, A~! = At and so 
AAt = AA! = AIA = ATA 


Finally, we note that, if A is normal then so too is its inverse A~!, since 


A"(AT!)T = AMAT)! = (ATA)! = (AAT)! = (AT TAT = (ATTA 


This broad class of matrices is important in the discussion of eigenvectors and 
eigenvalues in the next section. 


8.13 Eigenvectors and eigenvalues 


Suppose that a linear operator A transforms vectors x in an N-dimensional 
vector space into other vectors Ax in the same space. The possibility then arises 
that there exist vectors x each of which is transformed by A into a multiple of 
itself. Such vectors would have to satisfy 


Ax = x. (8.67) 


Any non-zero vector x that satisfies (8.67) for some value of 4 is called an 
eigenvector of the linear operator A, and 4 is called the corresponding eigenvalue. 
As will be discussed below, in general the operator A has N independent 
eigenvectors x’, with eigenvalues /;. The 4; are not necessarily all distinct. 

If we choose a particular basis in the vector space, we can write (8.67) in terms 
of the components of A and x with respect to this basis as the matrix equation 


Ax = 1x, (8.68) 


where A is an N x N matrix. The column matrices x that satisfy (8.68) obviously 
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represent the eigenvectors x of A in our chosen coordinate system. Convention- 
ally, these column matrices are also referred to as the eigenvectors of the matrix 
AS Clearly, if x is an eigenvector of A (with some eigenvalue 4) then any scalar 
multiple x is also an eigenvector with the same eigenvalue. We therefore often 
use normalised eigenvectors, for which 


x'x=1 


(note that x*x corresponds to the inner product (x|x) in our basis). Any eigen- 
vector x can be normalised by dividing all its components by the scalar (x'x)!/. 

As will be seen, the problem of finding the eigenvalues and corresponding 
eigenvectors of a square matrix A plays an important role in many physical 
investigations. Throughout this chapter we denote the ith eigenvector of a square 
matrix A by x! and the corresponding eigenvalue by /;. This superscript notation 
for eigenvectors is used to avoid any confusion with components. 


> A non-singular matrix A has eigenvalues A; and eigenvectors x'. Find the eigenvalues and 
eigenvectors of the inverse matrix Aq. 


The eigenvalues and eigenvectors of A satisfy 
Ax! = jx. 
Left-multiplying both sides of this equation by A7!, we find 
ATAX' = A)ATIX’. 
Since A~'A = I, on rearranging we obtain 
F ile 
Ax! = —x'. 
Thus, we see that A~' has the same eigenvectors x' as does A, but the corresponding 
eigenvalues are 1/A;. < 
In the remainder of this section we will discuss some useful results concerning 
the eigenvectors and eigenvalues of certain special (though commonly occurring) 
square matrices. The results will be established for matrices whose elements may 
be complex; the corresponding properties for real matrices may be obtained as 
special cases. 


8.13.1 Eigenvectors and eigenvalues of a normal matrix 


In subsection 8.12.7 we defined a normal matrix A as one that commutes with its 
Hermitian conjugate, so that 


ATA=AA. 


S In this context, when referring to linear combinations of eigenvectors x we will normally use the 
term ‘vector’. 
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We also showed that both Hermitian and unitary matrices (or symmetric and 
orthogonal matrices in the real case) are examples of normal matrices. We now 
discuss the properties of the eigenvectors and eigenvalues of a normal matrix. 

If x is an eigenvector of a normal matrix A with corresponding eigenvalue 1 
then Ax = 1x, or equivalently, 


(A—Al)x =0. (8.69) 


Denoting B = A—Al, (8.69) becomes Bx = 0 and, taking the Hermitian conjugate, 
we also have 


(Bx)' = xtBt =0. (8.70) 
From (8.69) and (8.70) we then have 
x'BIBx =0. (8.71) 


However, the product B‘B is given by 


BYB = (A— Alt (A— Al) = (AT — AIA — Al) = ATA — A — DAT 4 2". 


Now since A is normal, AAt = A‘A and so 


BIB = AAT — 7° A—AAT + As = (A—AN(A— Al) = BBI, 
and hence B is also normal. From (8.71) we then find 
x' Bt Bx = x'BBIx = (Bix)'BIx = 0, 
from which we obtain 
BYx = (At —2°1I)x = 0. 


Therefore, for a normal matrix A, the eigenvalues of A‘ are the complex conjugates 
of the eigenvalues of A. 

Let us now consider two eigenvectors x' and x/ of a normal matrix A corre- 
sponding to two different eigenvalues J; and /;. We then have 


Ax! = 1jx’, (8.72) 
Ax! = Ajx!, (8.73) 

Multiplying (8.73) on the left by (x')' we obtain 
(xi)TAxd = Aj(xi)hx/. (8.74) 


However, on the LHS of (8.74) we have 


(xTA = (ATx)T = AF x')h = A(x, (8.75) 


where we have used (8.40) and the property just proved for a normal matrix to 
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write Atx! = J;x', From (8.74) and (8.75) we have 
(A; — 4;)(x')Tx? = 0. (8.76) 


Thus, if 4; # 4; the eigenvectors x' and x/ must be orthogonal, ie. (x')'x/ = 0. 

It follows immediately from (8.76) that if all N eigenvalues of a normal matrix 
A are distinct then all N eigenvectors of A are mutually orthogonal. If, however, 
two or more eigenvalues are the same then further consideration is required. An 
eigenvalue corresponding to two or more different eigenvectors (i.e. they are not 
simply multiples of one another) is said to be degenerate. Suppose that /, is k-fold 
degenerate, Le. 


Ax'=A,x' for i= 1,2,...,k, (8.77) 


but that it is different from any of Agai, 2x42, etc. Then any linear combination 
of these x! is also an eigenvector with eigenvalue A, since, for z = ey cix', 


k k k 
Az = AS cx = S cj AX' = Se, cA x! = yz. (8.78) 
i=l i=l i=l 


If the x! defined in (8.77) are not already mutually orthogonal then we can 
construct new eigenvectors z! that are orthogonal by the following procedure: 


ee [aye] 32 [ante] 3! 
zk 2 a [ays] as [ay] AG 


In this procedure, known as Gram—Schmidt orthogonalisation, each new eigen- 
vector z! is normalised to give the unit vector 2! before proceeding to the construc- 
tion of the next one (the normalisation is carried out by dividing each element of 
the vector z! by [(z')'z‘]/*). Note that each factor in brackets (2”)'x" is a scalar 
product and thus only a number. It follows that, as shown in (8.78), each vector 
z' so constructed is an eigenvector of A with eigenvalue 2; and will remain so 
on normalisation. It is straightforward to check that, provided the previous new 
eigenvectors have been normalised as prescribed, each z' is orthogonal to all its 
predecessors. (In practice, however, the method is laborious and the example in 
subsection 8.14.1 gives a less rigorous but considerably quicker way.) 

Therefore, even if A has some degenerate eigenvalues we can by construction 
obtain a set of N mutually orthogonal eigenvectors. Moreover, it may be shown 
(although the proof is beyond the scope of this book) that these eigenvectors 
are complete in that they form a basis for the N-dimensional vector space. As 
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a result any arbitrary vector y can be expressed as a linear combination of the 
eigenvectors x': 


N 
y= Soax', (8.79) 
i=1 
where a; = (x')'y. Thus, the eigenvectors form an orthogonal basis for the vector 
space. By normalising the eigenvectors so that (x')'x' = 1 this basis is made 
orthonormal. 


> Show that a normal matrix A can be written in terms of its eigenvalues 4; and orthonormal 
eigenvectors x' as 


N 
A=)" Axi’). (8.80) 
i=1 


The key to proving the validity of (8.80) is to show that both sides of the expression give 
the same result when acting on an arbitary vector y. Since A is normal, we may expand y 
in terms of the eigenvectors x’, as shown in (8.79). Thus, we have 


N N 
Ay =A x: ax! => a ajAix'. 
i=1 i=1 


Alternatively, the action of the RHS of (8.80) on y is given by 


N N 
DLA OD y = DF aidex, 
i=l i=l 


since a; = (x')'y. We see that the two expressions for the action of each side of (8.80) on y 
are identical, which implies that this relationship is indeed correct. < 


8.13.2 Eigenvectors and eigenvalues of Hermitian and anti-Hermitian matrices 


For a normal matrix we showed that if Ax = 2x then A*x = A*x. However, if A is 
also Hermitian, A = A‘, it follows necessarily that 2 = 2°. Thus, the eigenvalues 
of an Hermitian matrix are real, a result which may be proved directly. 


> Prove that the eigenvalues of an Hermitian matrix are real. 


For any particular eigenvector x', we take the Hermitian conjugate of Ax' = 1;x! to give 
(xJTAT = AR (xt (8.81) 


Using A‘ = A, since A is Hermitian, and multiplying on the right by x’, we obtain 
(x)TAX! = Aj (x!)Tx!, (8.82) 
But multiplying Ax' = 2,x' through on the left by (x') gives 
(x!JTAx! = Ai(x))*x!, 
Subtracting this from (8.82) yields 
0 = (AF — Adx'tx! 
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But (x')'x' is the modulus squared of the non-zero vector x' and is thus non-zero. Hence 
A; must equal 4; and thus be real. The same argument can be used to show that the 
eigenvalues of a real symmetric matrix are themselves real. < 


The importance of the above result will be apparent to any student of quantum 
mechanics. In quantum mechanics the eigenvalues of operators correspond to 
measured values of observable quantities, e.g. energy, angular momentum, parity 
and so on, and these clearly must be real. If we use Hermitian operators to 
formulate the theories of quantum mechanics, the above property guarantees 
physically meaningful results. 

Since an Hermitian matrix is also a normal matrix, its eigenvectors are orthog- 
onal (or can be made so using the Gram—Schmidt orthogonalisation procedure). 
Alternatively we can prove the orthogonality of the eigenvectors directly. 


> Prove that the eigenvectors corresponding to different eigenvalues of an Hermitian matrix 
are orthogonal. 


Consider two unequal eigenvalues 4; and J; and their corresponding eigenvectors satisfying 
Ax! = 1x‘, (8.83) 
Ax! = Ajx/. (8.84) 
Taking the Hermitian conjugate of (8.83) we find (x‘)fAT = A°(x')t. Multiplying this on the 
right by x’ we obtain 
(x TATX! = A (xii, 
and similarly multiplying (8.84) through on the left by (x')' we find 
(xi TAX! = 2j(x')Tx/. 
Then, since A‘ = A, the two left-hand sides are equal and, because the A; are real, on 
subtraction we obtain 


0 = (A; — A)(x')tx’, 

Finally we note that 4; # A; and so (x')'x/ = 0, ie. the eigenvectors x’ and x/ are 
orthogonal. < 

In the case where some of the eigenvalues are equal, further justification of the 
orthogonality of the eigenvectors is needed. The Gram—Schmidt orthogonalisa- 
tion procedure discussed above provides a proof of, and a means of achieving, 
orthogonality. The general method has already been described and we will not 
repeat it here. 

We may also consider the properties of the eigenvalues and eigenvectors of an 
anti-Hermitian matrix, for which At = —A and thus 


AAt = A(—A) = (—A)A = ATA. 


Therefore matrices that are anti-Hermitian are also normal and so have mutu- 
ally orthogonal eigenvectors. The properties of the eigenvalues are also simply 
deduced, since if Ax = 2x then 


2x = Ax = —Ax = — Ax. 
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Hence 2° = —/ and so 4 must be pure imaginary (or zero). In a similar manner 
to that used for Hermitian matrices, these properties may be proved directly. 


8.13.3 Eigenvectors and eigenvalues of a unitary matrix 


A unitary matrix satisfies A‘ = A! and is also a normal matrix, with mutually 
orthogonal eigenvectors. To investigate the eigenvalues of a unitary matrix, we 
note that if Ax = Ax then 


x'x = xt ATAx = 2° Ax'x, 


29% 4 


and we deduce that 42" = |A|* = 1. Thus, the eigenvalues of a unitary matrix 
have unit modulus. 


8.13.4 Eigenvectors and eigenvalues of a general square matrix 


When an N x N matrix is not normal there are no general properties of its 
eigenvalues and eigenvectors; in general it is not possible to find any orthogonal 
set of N eigenvectors or even to find pairs of orthogonal eigenvectors (except 
by chance in some cases). While the N non-orthogonal eigenvectors are usually 
linearly independent and hence form a basis for the N-dimensional vector space, 
this is not necessarily so. It may be shown (although we will not prove it) that any 
N x N matrix with distinct eigenvalues has N linearly independent eigenvectors, 
which therefore form a basis for the N-dimensional vector space. If a general 
square matrix has degenerate eigenvalues, however, then it may or may not have 
N linearly independent eigenvectors. A matrix whose eigenvectors are not linearly 
independent is said to be defective. 


8.13.5 Simultaneous eigenvectors 


We may now ask under what conditions two different normal matrices can have 
a common set of eigenvectors. The result — that they do so if, and only if, they 
commute — has profound significance for the foundations of quantum mechanics. 

To prove this important result let A and B be two N x N normal matrices and 
x! be the ith eigenvector of A corresponding to eigenvalue 4;, i.e. 


Ax' =J;x' for i=1,2,...,N. 


For the present we assume that the eigenvalues are all different. 
(i) First suppose that A and B commute. Now consider 


ABx' = BAx! = BA;x! = /;Bx', 


where we have used the commutativity for the first equality and the eigenvector 
property for the second. It follows that A(Bx') = /,(Bx') and thus that Bx! is an 
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eigenvector of A corresponding to eigenvalue /;. But the eigenvector solutions of 
(A— Ajl)x! = 0 are unique to within a scale factor, and we therefore conclude that 


Bx! = Hx! 


for some scale factor y;. However, this is just an eigenvector equation for B and 
shows that x! is an eigenvector of B, in addition to being an eigenvector of A. By 
reversing the roles of A and B, it also follows that every eigenvector of B is an 
eigenvector of A. Thus the two sets of eigenvectors are identical. 

(ii) Now suppose that A and B have all their eigenvectors in common, a typical 
one x’ satisfying both 


Ax'=1;x' and Bx' = mix! 


As the eigenvectors span the N-dimensional vector space, any arbitrary vector x 
in the space can be written as a linear combination of the eigenvectors, 


N 
X= y CiX'. 
i=l 


Now consider both 
N N N 
ABx = AB 5 ax =A > Cipix’ = ys: Ci: Lui’, 
i=l i=l i=l 


and 
N N N 
BAx = BA » cix'=B a ciAix' = a CiptidiX' 
i=l i=l i=1 


It follows that ABx and BAx are the same for any arbitrary x and hence that 
(AB — BA)x = 0 


for all x. That is, A and B commute. 

This completes the proof that a necessary and sufficient condition for two 
normal matrices to have a set of eigenvectors in common is that they commute. 
It should be noted that if an eigenvalue of A, say, is degenerate then not all of 
its possible sets of eigenvectors will also constitute a set of eigenvectors of B. 
However, provided that by taking linear combinations one set of joint eigenvectors 
can be found, the proof is still valid and the result still holds. 

When extended to the case of Hermitian operators and continuous eigenfunc- 
tions (sections 17.2 and 17.3) the connection between commuting matrices and 
a set of common eigenvectors plays a fundamental role in the postulatory basis 
of quantum mechanics. It draws the distinction between commuting and non- 
commuting observables and sets limits on how much information about a system 
can be known, even in principle, at any one time. 
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8.14 Determination of eigenvalues and eigenvectors 


The next step is to show how the eigenvalues and eigenvectors of a given N x N 
matrix A are found. To do this we refer to (8.68) and as in (8.69) rewrite it as 


Ax — Alx = (A— Al)x = 0. (8.85) 


The slight rearrangement used here is to write x as Ix, where | is the unit matrix 
of order N. The point of doing this is immediate since (8.85) now has the form 
of a homogeneous set of simultaneous equations, the theory of which will be 
developed in section 8.18. What will be proved there is that the equation Bx = 0 
only has a non-trivial solution x if |B| = 0. Correspondingly, therefore, we must 
have in the present case that 


|A— Al] =0, (8.86) 


if there are to be non-zero solutions x to (8.85). 

Equation (8.86) is known as the characteristic equation for A and its LHS as 
the characteristic or secular determinant of A. The equation is a polynomial of 
degree N in the quantity 4. The N roots of this equation /;, i = 1,2,...,N, give 
the eigenvalues of A. Corresponding to each /; there will be a column vector x’, 
which is the ith eigenvector of A and can be found by using (8.68). 

It will be observed that when (8.86) is written out as a polynomial equation in 
A, the coefficient of —A‘—! in the equation will be simply Ay, + Ay +---+ Ayn 
relative to the coefficient of 2%. As discussed in section 8.8, the quantity )7”, Ai 
is the trace of A and, from the ordinary theory of polynomial equations, will be 
equal to the sum of the roots of (8.86): 


N 
SoA =TrA. (8.87) 
i=1 


This can be used as one check that a computation of the eigenvalues 2; has been 
done correctly. Unless equation (8.87) is satisfied by a computed set of eigenvalues, 
they have not been calculated correctly. However, that equation (8.87) is satisfied is 
a necessary, but not sufficient, condition for a correct computation. An alternative 
proof of (8.87) is given in section 8.16. 


> Find the eigenvalues and normalised eigenvectors of the real symmetric matrix 


1G et! 3 
A={ 1 1 -3 |. 
3 -3 -3 


Using (8.86), 
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Expanding out this determinant gives 


(1—A) [0 — A)(—3 — 4) — (—3)(—3)] + 1[(-—3)(3) — 1-3 — 4)] 
+ 3[1(—3) — (1 — 4)(3)] = 0, 


which simplifies to give 
(1 — Ay? + 24 — 12) + (A— 6) + 3(32 — 6) = 0, 
=> (A—2)A—3)A +6) =0. 


Hence the roots of the characteristic equation, which are the eigenvalues of A, are 2; = 2, 
Ay = 3, 24; = —6. We note that, as expected, 


Ay tag +43 =—-1=14+1-3=Ay + Ay +A33 = TA. 


For the first root, A; = 2, a suitable eigenvector x', with elements x1, x2, x3, must satisfy 
Ax! = 2x! or, equivalently, 
X1 + x2 + 3x3 = 2x1, 
X47 xX. 3x3 = 2x, (8.88) 
3x1 _ 3x2 = 3x3 = 2x3. 


These three equations are consistent (to ensure this was the purpose in finding the particular 
values of 4) and yield x3 = 0, x1 = x2 =k, where k is any non-zero number. A suitable 
eigenvector would thus be 


x'=(k k Oj. 
If we apply the normalisation condition, we require k* + k? + 0? = 1 or k = 1/,/2. Hence 
1 1 Tod 
v= (5 — 0) =—(1 1 0). 
Be eos 


Repeating the last paragraph, but with the factor 2 on the RHS of (8.88) replaced 
successively by A, = 3 and A; = —6, gives two further normalised eigenvectors 


~=—(1 -1 1), x=—(1 -1 —-2)'.<« 


In the above example, the three values of A are all different and A is a 
real symmetric matrix. Thus we expect, and it is easily checked, that the three 
eigenvectors are mutually orthogonal, ie. 


(x!) "x? = (x!) "x3 = (xP) "x3 =0. 


It will be apparent also that, as expected, the normalisation of the eigenvectors 
has no effect on their orthogonality. 


8.14.1 Degenerate eigenvalues 


We return now to the case of degenerate eigenvalues, i.e. those that have two or 
more associated eigenvectors. We have shown already that it is always possible 
to construct an orthogonal set of eigenvectors for a normal matrix, see subsec- 
tion 8.13.1, and the following example illustrates one method for constructing 
such a set. 
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> Construct an orthonormal set of eigenvectors for the matrix 


ROSES 
A=[{ 0 —2 0 
3 @ il 
We first determine the eigenvalues using |A — Al| = 0: 
1-4 0 3 
O=| 0 -2-4 O | =—(1—4)(2 +A) +3(3)(2 +4) 
3 0 1-A 
= (4—A\(A4+ 2). 
Thus A, = 4, 22 = —2 = 13. The eigenvector x! = (x, x2 x3)" is found from 
1 0 3 X1 x1 1 1 
0 —2 0 x2 =4 x2 => x! = 0 
ay oA x3 x3 V2\ 1 
A general column vector that is orthogonal to x! is 
x=(a b —a)', (8.89) 


and it is easily shown that 


1 0 3 a a 
Ax=|{ 0 —2 0 b =-—2 b = —2x, 
3 0 1 —a —a 


Thus x is a eigenvector of A with associated eigenvalue —2. It is clear, however, that there 
is an infinite set of eigenvectors x all possessing the required property; the geometrical 
analogue is that there are an infinite number of corresponding vectors x lying in the 
plane that has x! as its normal. We do require that the two remaining eigenvectors are 
orthogonal to one another, but this still leaves an infinite number of possibilities. For x’, 
therefore, let us choose a simple form of (8.89), suitably normalised, say, 


xr=(0 1 O/T. 
The third eigenvector is then specified (to within an arbitrary multiplicative constant) 


by the requirement that it must be orthogonal to x! and x’; thus x? may be found by 
evaluating the vector product of x! and x? and normalising the result. This gives 


~=—(-1 0 1, 


to complete the construction of an orthonormal set of eigenvectors. < 


8.15 Change of basis and similarity transformations 


Throughout this chapter we have considered the vector x as a geometrical quantity 
that is independent of any basis (or coordinate system). If we introduce a basis 
e;, i = 1,2,...,N, into our N-dimensional vector space then we may write 


X = Xjey + X2€2 +--+ + XNEN, 
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and represent x in this basis by the column matrix 


X=(Xp XQ-7° Xn)» 
having components x;. We now consider how these components change as a result 
of a prescribed change of basis. Let us introduce a new basis ej, i = 1,2,...,.N, 
which is related to the old basis by 


N 
e = > Syei, (8.90) 
i=1 


the coefficient S;; being the ith component of e, with respect to the old (unprimed) 
basis. For an arbitrary vector x it follows that 


N N 
ae a / 
x= y xe; = y xe; = x; y Sijei- 
i=l j=l 


j= j=l i=1 


From this we derive the relationship between the components of x in the two 
coordinate systems as 


N 
/ 
xi= ) SijX}> 
j=l 
which we can write in matrix form as 


x = Sx’ (8.91) 


where S is the transformation matrix associated with the change of basis. 
Furthermore, since the vectors e’, are linearly independent, the matrix S is 
non-singular and so possesses an inverse S~'. Multiplying (8.91) on the left by 


S~! we find 
x =S!x, (8.92) 


which relates the components of x in the new basis to those in the old basis. 
Comparing (8.92) and (8.90) we note that the components of x transform inversely 
to the way in which the basis vectors e; themselves transform. This has to be so, 
as the vector x itself must remain unchanged. 

We may also find the transformation law for the components of a linear 
operator under the same change of basis. Now, the operator equation y = Ax 
(which is basis independent) can be written as a matrix equation in each of the 
two bases as 


y = Ax, y =A’. (8.93) 
But, using (8.91), we may rewrite the first equation as 
Sy’=ASx = y =S'ASy’. 
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Comparing this with the second equation in (8.93) we find that the components 
of the linear operator A transform as 


A’ =S"IAS. (8.94) 


Equation (8.94) is an example of a similarity transformation — a transformation 
that can be particularly useful in converting matrices into convenient forms for 
computation. 

Given a square matrix A, we may interpret it as representing a linear operator 
A in a given basis e;. From (8.94), however, we may also consider the matrix 
A’ = S'AS, for any non-singular matrix S, as representing the same linear 
operator A but in a new basis e’, related to the old basis by 


’ ., 
ej = Sij@i- 
i 


Therefore we would expect that any property of the matrix A that represents 
some (basis-independent) property of the linear operator A will also be shared 
by the matrix A’. We list these properties below. 


(i) If A=1 then A’ = |, since, from (8.94), 
A =S1iS=S'S=1. (8.95) 
(ii) The value of the determinant is unchanged: 
|A’| = |S“'AS| = |S“'/A||S| = |Al|S“||S| = |AlIS~'S| = Al. (8.96) 


(iii) The characteristic determinant and hence the eigenvalues of A’ are the 
same as those of A: from (8.86), 


|A’ — All =|S~!AS — Al] = |S~1(A — ANs| 
=|S-!||S||A — Al] = |A — All. (8.97) 


(iv) The value of the trace is unchanged: from (8.87), 


TrA’ = So Ai = Se Ss SB yA peSei 
; i jk 
= S05 SulS Dy Ap = 5S oA = >> Ay 
i oj ok k j 


Jj 
=TrA. (8.98) 


An important class of similarity transformations is that for which S is a uni- 
tary matrix; in this case A’ = S-'AS = S'AS. Unitary transformation matrices 
are particularly important, for the following reason. If the original basis e; is 
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orthonormal and the transformation matrix S is unitary then 
(ejle}) = ‘oy Suite] S,je;) 
k r 
= >> Si >> S,jfexle) 
k r 
= Si DE SiSkr = YF SiiSej = (S*S)iy = Sip 
k r k 


showing that the new basis is also orthonormal. 
Furthermore, in addition to the properties of general similarity transformations, 
for unitary transformations the following hold. 


(i) If A is Hermitian (anti-Hermitian) then A’ is Hermitian (anti-Hermitian), 
ie. if At = +A then 


(A’)' = (StAS)T = Stats = +STAS = +A’. (8.99) 


(ii) If A is unitary (so that At = A7!) then A’ is unitary, since 


(A’)tA’ = (STAS)'(STAS) = STATSSTAS = STATAS 
=SiIS=1. (8.100) 


8.16 Diagonalisation of matrices 


Suppose that a linear operator A is represented in some basis e;, i = 1,2,...,.N, 
by the matrix A. Consider a new basis x/ given by 


N 
xi = y, Sij@i, 
i=l 
where the x/ are chosen to be the eigenvectors of the linear operator A, ie. 
Axi = 1;x!. (8.101) 


In the new basis, A is represented by the matrix A’ = S~!AS, which has a 
particularly simple form, as we shall see shortly. The element S;; of S is the ith 
component, in the old (unprimed) basis, of the jth eigenvector x/ of A, ie. the 
columns of S are the eigenvectors of the matrix A: 


tT T t 
S= xb x2 22. XN 5 
hc 1 
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that is, S;; = (x/);. Therefore A’ is given by 
(S~'AS)ij = Dy SoS) AeiS1j 
ee! 
= ST SoS ied”) 
ko 
= SOS) adie) 
k 


7 Dy Ay(S™" ik Sey = A6i7- 
k 


So the matrix A’ is diagonal with the eigenvalues of A as the diagonal elements, 
Le. 


Ay 0 es 0 

N= 0 Ar : 
or 0 

0 O° Fs 


Therefore, given a matrix A, if we construct the matrix S that has the eigen- 
vectors of A as its columns then the matrix A’ = S~'AS is diagonal and has the 
eigenvalues of A as its diagonal elements. Since we require S to be non-singular 
(|S| # 0), the N eigenvectors of A must be linearly independent and form a basis 
for the N-dimensional vector space. It may be shown that any matrix with distinct 
eigenvalues can be diagonalised by this procedure. If, however, a general square 
matrix has degenerate eigenvalues then it may, or may not, have N linearly 
independent eigenvectors. If it does not then it cannot be diagonalised. 

For normal matrices (which include Hermitian, anti-Hermitian and unitary 
matrices) the N eigenvectors are indeed linearly independent. Moreover, when 
normalised, these eigenvectors form an orthonormal set (or can be made to do 
so). Therefore the matrix S with these normalised eigenvectors as columns, ie. 
whose elements are S;; = (x/);, has the property 


(S'S), = S\(S (Sky = Do SiS = DLC = OY Oi 
k 


Hence S$ is unitary (S~! = S') and the original matrix A can be diagonalised by 
A’ =S"'AS =STAS. 


Therefore, any normal matrix A can be diagonalised by a similarity transformation 
using a unitary transformation matrix S. 
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> Diagonalise the matrix 


The matrix A is symmetric and so may be diagonalised by a transformation of the form 
A’ = S'AS, where S has the normalised eigenvectors of A as its columns. We have already 
found these eigenvectors in subsection 8.14.1, and so we can write straightaway 


1 1 oO -tl 

S=—=/|0 2 0 

v2\1 0 1 
We note that although the eigenvalues of A are degenerate, its three eigenvectors are 
linearly independent and so A can still be diagonalised. Thus, calculating S'AS we obtain 


pf eee 1 0 3 1 0 -1 
STAS = S38 V2 0 0 -2 0 0 2 0 
-1 0 1 3 0 1 1 0 1 


4 0 0 
=|0 2 0 |, 
0 0 -2 


which is diagonal, as required, and has as its diagonal elements the eigenvalues of A. < 


If a matrix A is diagonalised by the similarity transformation A’ = S~'AS, so 
that A’ = diag(/), 42,..., 2), then we have immediately 


N 
TrA =TrA= a his (8.102) 
i=1 
N 
IA =|Al =] [a (8.103) 
i=1 


since the eigenvalues of the matrix are unchanged by the transformation. More- 
over, these results may be used to prove the rather useful trace formula 


|exp A| = exp(Tr A), (8.104) 
where the exponential of a matrix is as defined in (8.38). 


> Prove the trace formula (8.104). 


At the outset, we note that for the similarity transformation A’ = S~'AS, we have 
(A’)" = (S-!AS)(S“!AS)---(S~!AS) = S“1A"S. 
Thus, from (8.38), we obtain exp A’ = S~!(exp A)S, from which it follows that |exp A’| = 
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|exp A]. Moreover, by choosing the similarity transformation so that it diagonalises A, we 
have A’ = diag(A1, 42,...,4n), and so 
x 
|exp A] =| expA’| =| exp[diag(41, 42,...,A)]] = |diag(exp 41, exp 4a,...,expAy)| =| | exp 4. 
i=1 


Rewriting the final product of exponentials of the eigenvalues as the exponential of the 
sum of the eigenvalues, we find 


N N 
jexp A| = Il exp A; = exp es i) = exp(Tr A), 


i=l i=1 
which gives the trace formula (8.104). < 


8.17 Quadratic and Hermitian forms 


Let us now introduce the concept of quadratic forms (and their complex ana- 
logues, Hermitian forms). A quadratic form Q is a scalar function of a real vector 
x given by 


Q(x) = (x|Ax), (8.105) 


for some real linear operator A. In any given basis (coordinate system) we can 
write (8.105) in matrix form as 


Q(x) = xT Ax, (8.106) 


where A is a real matrix. In fact, as will be explained below, we need only consider 
the case where A is symmetric, ic. A= A’. As an example in a three-dimensional 
space, 


1 1 3 x1 
Q= xT Ax = (a x2 xs) 1 1 -3 x2 
3 -3 -3 X3 

= xt + x5 _ 3x3 + 2x1xX2 + 6x1x3 — 6X23. (8.107) 


It is reasonable to ask whether a quadratic form Q = x™Mx, where M is any 
(possibly non-symmetric) real square matrix, is a more general definition. That 
this is not the case may be seen by expressing M in terms of a symmetric matrix 
A= 5(M+M"') and an antisymmetric matrix B = 5(M—M') such that M = A+B. 
We then have 


Q =x'Mx = x'Ax + x7 Bx. (8.108) 
However, Q is a scalar quantity and so 


Q= gr = (xTAx)" oF (x™Bx)" = x'A'x+x'Blx = x’ Ax — x’ Bx. 
(8.109) 


Comparing (8.108) and (8.109) shows that x™Bx = 0, and hence x'Mx = x'Ax, 
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i.e. Q is unchanged by considering only the symmetric part of M. Hence, with no 
loss of generality, we may assume A = A! in (8.106). 

From its definition (8.105), Q is clearly a basis- (i.e. coordinate-) independent 
quantity. Let us therefore consider a new basis related to the old one by an 
orthogonal transformation matrix S, the components in the two bases of any 
vector x being related (as in (8.91)) by x = Sx’ or, equivalently, by x’ = S~!x = 
Sx. We then have 


OQ =x'Ax = (x’)'STASX’ = (x')TA’X’, 


where (as expected) the matrix describing the linear operator A in the new 
basis is given by A’ = S'AS (since S' = S~!). But, from the last section, if we 
choose as S the matrix whose columns are the normalised eigenvectors of A then 
A’ = STAS is diagonal with the eigenvalues of A as the diagonal elements. (Since 
A is symmetric, its normalised eigenvectors are orthogonal, or can be made so, 
and hence § is orthogonal with $~! = S".) 

In the new basis 


O =xTAx = (x))TAX! = Ax? + doxy? be FAN xy, (8.110) 


where A = diag(A;, 42,...,2) and the /; are the eigenvalues of A. It should be 
noted that Q contains no cross-terms of the form x}x}. 


> Find an orthogonal transformation that takes the quadratic form (8.107) into the form 


Ny SD a Pee 
AtX, Hb A2xXy +43X3°. 


The required transformation matrix S has the normalised eigenvectors of A as its columns. 
We have already found these in section 8.14, and so we can write immediately 


igs ape. 4 
$22 Bh cat) | 
a\y FS 


which is easily verified as being orthogonal. Since the eigenvalues of A are 2 = 2, 3, and 
—6, the general result already proved shows that the transformation x = Sx’ will carry 
(8.107) into the form Dx? + 3x/7 - 6x)". This may be verified most easily by writing out 
the inverse transformation x’ = S~!x = S'x and substituting. The inverse equations are 


x1 = (x1 + 2)/-V2, 
x) = (x1 — x2 + x3)/V3, (8.111) 
x4 = (x1 — x) — 2x3)/6. 


If these are substituted into the form Q = 2x? + 3x} — 6x4? then the original expression 
(8.107) is recovered. < 


In the definition of Q it was assumed that the components x1, x2, x3 and the 
matrix A were real. It is clear that in this case the quadratic form Q = x" Ax is real 
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also. Another, rather more general, expression that is also real is the Hermitian 
form 


H(x) = xt Ax, (8.112) 


where A is Hermitian (ie. A‘ = A) and the components of x may now be complex. 
It is straightforward to show that H is real, since 


H* =(H') =x'tatx =xtax = H. 


With suitable generalisation, the properties of quadratic forms apply also to Her- 
mitian forms, but to keep the presentation simple we will restrict our discussion 
to quadratic forms. 

A special case of a quadratic (Hermitian) form is one for which Q = x™Ax 
is greater than zero for all column matrices x. By choosing as the basis the 
eigenvectors of A we have Q in the form 


Q = Aixt a Aox5 i 3X}. 


The requirement that Q > 0 for all x means that all the eigenvalues 1; of A must 
be positive. A symmetric (Hermitian) matrix A with this property is called positive 
definite. If, instead, Q > 0 for all x then it is possible that some of the eigenvalues 
are zero, and A is called positive semi-definite. 


8.17.1 The stationary properties of the eigenvectors 


Consider a quadratic form, such as Q(x) = (x|Ax), equation (8.105), in a fixed 
basis. As the vector x is varied, through changes in its three components x1, x2 
and x3, the value of the quantity Q also varies. Because of the homogeneous 
form of Q we may restrict any investigation of these variations to vectors of unit 
length (since multiplying any vector x by any scalar k simply multiplies the value 
of Q by a factor k?). 

Of particular interest are any vectors x that make the value of the quadratic 
form a maximum or minimum. A necessary, but not sufficient, condition for this 
is that Q is stationary with respect to small variations Ax in x, whilst (x|x) is 
maintained at a constant value (unity). 

In the chosen basis the quadratic form is given by Q = x'Ax and, using 
Lagrange undetermined multipliers to incorporate the variational constraints, we 
are led to seek solutions of 


A[xTAx — A(x'x — 1)] = 0. (8.113) 


This may be used directly, together with the fact that (Ax')Ax = xTA Ax, since A 
is symmetric, to obtain 


Ax = Ax (8.114) 
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as the necessary condition that x must satisfy. If (8.114) is satisfied for some 
eigenvector x then the value of Q(x) is given by 


QO =x'Ax =xldx=2. (8.115) 


However, if x and y are eigenvectors corresponding to different eigenvalues then 
they are (or can be chosen to be) orthogonal. Consequently the expression y’ Ax 
is necessarily zero, since 


y' Ax = y! dx = dy'x = 0. (8.116) 


Summarising, those column matrices x of unit magnitude that make the 
quadratic form Q stationary are eigenvectors of the matrix A, and the stationary 
value of Q is then equal to the corresponding eigenvalue. It is straightforward 
to see from the proof of (8.114) that, conversely, any eigenvector of A makes Q 
stationary. 

Instead of maximising or minimising Q = x'Ax subject to the constraint 
xx = 1, an equivalent procedure is to extremise the function 


xT Ax 


x!x 


A(x) = 


> Show that if A(x) is stationary then x is an eigenvector of A and 2(x) is equal to the 
corresponding eigenvalue. 


We require AA(x) = 0 with respect to small variations in x. Now 


AA = = [(x™x) (AxTAx + x™A Ax) — xTAx (Ax™x + xTAx) | 


(x™x)? 
_ 2AxTAx , x'Ax)\ Ax'x 
~ xTx xx xTx ? 


since x'A Ax = (Ax')Ax and x'Ax = (Ax')x. Thus 
Ad = z Ax! [Ax — 2(x)x] 
b= Say A ; 
Hence, if AA = 0 then Ax = A(x)x, Le. x is an eigenvector of A with eigenvalue A(x). < 


Thus the eigenvalues of a symmetric matrix A are the values of the function 


at its stationary points. The eigenvectors of A lie along those directions in space 
for which the quadratic form Q = x'Ax has stationary values, given a fixed 
magnitude for the vector x. Similar results hold for Hermitian matrices. 
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8.17.2 Quadratic surfaces 


The results of the previous subsection may be turned round to state that the 
surface given by 


xT Ax = constant = 1 (say) (8.117) 


and called a quadratic surface, has stationary values of its radius (i.e. origin— 
surface distance) in those directions that are along the eigenvectors of A. More 
specifically, in three dimensions the quadratic surface x'Ax = 1 has its principal 
axes along the three mutually perpendicular eigenvectors of A, and the squares 
of the corresponding principal radii are given by /;!, i = 1,2,3. As well as 
having this stationary property of the radius, a principal axis is characterised by 
the fact that any section of the surface perpendicular to it has some degree of 
symmetry about it. If the eigenvalues corresponding to any two principal axes are 
degenerate then the quadratic surface has rotational symmetry about the third 
principal axis and the choice of a pair of axes perpendicular to that axis is not 
uniquely defined. 


> Find the shape of the quadratic surface 


xt — 3e _ 3x3 + 2x1x2 + 6x1x3 — 6X2x3 = 1. 


If, instead of expressing the quadratic surface in terms of x1, x2, x3, as in (8.107), we 
were to use the new variables x/, x}, x4 defined in (8.111), for which the coordinate axes 
are along the three mutually perpendicular eigenvector directions (1,1,0), (1,—1,1) and 
(1,—1,—2), then the equation of the surface would take the form (see (8.110)) 

aie xe xe 


+ a = 1. 
(1/V2? 1/3 (1/6)? 
Thus, for example, a section of the quadratic surface in the plane x, = 0, Le. x1; — x. — 
2x3 = 0, is an ellipse, with semi-axes 1/./2 and 1/./3. Similarly a section in the plane 
x =X; +X =0 is a hyperbola. < 


Clearly the simplest three-dimensional situation to visualise is that in which all 
the eigenvalues are positive, since then the quadratic surface is an ellipsoid. 


8.18 Simultaneous linear equations 


In physical applications we often encounter sets of simultaneous linear equations. 


In general we may have M equations in N unknowns x1, X2,...,xy of the form 
Ajtx1 + Ajax. +--+ + Ainxn = bi, 
Ax + Arx2 +++ + Aanxn = by, 
(8.118) 


AmixX1 + Am2X2 +--+: +Amnxn = bm, 
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where the Aj; and b; have known values. If all the b; are zero then the system of 
equations is called homogeneous, otherwise it is inhomogeneous. Depending on the 
given values, this set of equations for the N unknowns x1, x2, ..., Xv may have 
either a unique solution, no solution or infinitely many solutions. Matrix analysis 
may be used to distinguish between the possibilities. The set of equations may be 
expressed as a single matrix equation Ax = b, or, written out in full, as 


b 
Ay Ay... Ain x t 
Ay, An... Aon x2 by 
Amt Am2 --. Amn XN b 

M 


8.18.1 The range and null space of a matrix 


As we discussed in section 8.2, we may interpret the matrix equation Ax = b as 
representing, in some basis, the linear transformation Ax = b of a vector x in an 
N-dimensional vector space V into a vector b in some other (in general different) 
M-dimensional vector space W. 

In general the operator A will map any vector in V into some particular 
subspace of W, which may be the entire space. This subspace is called the range 
of A (or A) and its dimension is equal to the rank of A. Moreover, if A (and 
hence A) is singular then there exists some subspace of V that is mapped onto 
the zero vector 0 in W; that is, any vector y that lies in the subspace satisfies 
Ay = 0. This subspace is called the null space of A and the dimension of this 
null space is called the nullity of A. We note that the matrix A must be singular 
if M # N and may be singular even if M = N. 

The dimensions of the range and the null space of a matrix are related through 
the fundamental relationship 


rank A+ nullity A=N, (8.119) 


where N is the number of original unknowns x1, X2,...,XN- 


> Prove the relationship (8.119). 


As discussed in section 8.11, if the columns of an M x N matrix A are interpreted as the 
components, in a given basis, of N (M-component) vectors v1,¥2,...,Vy then rank A is 
equal to the number of linearly independent vectors in this set (this number is also equal 
to the dimension of the vector space spanned by these vectors). Writing (8.118) in terms 
of the vectors vj,V2,...,Vv, we have 


XV + X2V2 +--+ + Xn =b. (8.120) 


From this expression, we immediately deduce that the range of A is merely the span of 
the vectors v,,V2,...,Vy and hence has dimension r = rank A. 
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If a vector y lies in the null space of A then Ay = 0, which we may write as 
YiVi + y2Vo ++++ + yyw = 0. (8.121) 


As just shown above, however, only r (< N) of these vectors are linearly independent. By 
renumbering, if necessary, we may assume that v,,vo,...,v, form a linearly independent 
set; the remaining vectors, V,+1,V,+2,-.., Vv, can then be written as a linear superposition 
of v1, Vo,...,V,. We are therefore free to choose the N —r coefficients y,41, ¥-42,---. Yn 
arbitrarily and (8.121) will still be satisfied for some set of r coefficients yy, y2,...,y, (which 
are not all zero). The dimension of the null space is therefore N —r, and this completes 
the proof of (8.119). < 


Equation (8.119) has far-reaching consequences for the existence of solutions 
to sets of simultaneous linear equations such as (8.118). As mentioned previously, 
these equations may have no solution, a unique solution or infinitely many solutions. 
We now discuss these three cases in turn. 


No solution 


The system of equations possesses no solution unless b lies in the range of A; in 
this case (8.120) will be satisfied for some x1, x2,...,xn. This in turn requires the 
set of vectors b, v1, ¥2,...,Vv to have the same span (see (8.8)) as v1, V2,...,Vv. In 
terms of matrices, this is equivalent to the requirement that the matrix A and the 
augmented matrix 


Ait Ajr2 ... Ain by 

Ay An ... Aw by 
M= 

Ay: Amo --. Amn bu 


have the same rank r. If this condition is satisfied then b does lie in the range of 
A, and the set of equations (8.118) will have either a unique solution or infinitely 
many solutions. If, however, A and M have different ranks then there will be no 
solution. 


A unique solution 


If b lies in the range of A and if r = N then all the vectors vj, v2,..., vy in (8.120) 
are linearly independent and the equation has a unique solution x1,X2,...,XN- 


Infinitely many solutions 


If b lies in the range of A and if r < N then only r of the vectors vy, V2,...,¥y 
in (8.120) are linearly independent. We may therefore choose the coefficients of 
n—r vectors in an arbitrary way, while still satisfying (8.120) for some set of 
coefficients x1, X2,...,xXy. There are therefore infinitely many solutions, which span 
an (n—r)-dimensional vector space. We may also consider this space of solutions 
in terms of the null space of A: if x is some vector satisfying Ax = b and y is 
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any vector in the null space of A (1e. Ay = 0) then 
A(xt+y)=Ax+Ay=Ax+0=b, 


and so x+y is also a solution. Since the null space is (n — r)-dimensional, so too 
is the space of solutions. 

We may use the above results to investigate the special case of the solution of 
a homogeneous set of linear equations, for which b = 0. Clearly the set always has 
the trivial solution x; = x) =--- = x, = 0, and if r = N this will be the only 
solution. If r < N, however, there are infinitely many solutions; they form the 
null space of A, which has dimension n —r. In particular, we note that if M< N 
(i.e. there are fewer equations than unknowns) then r < N automatically. Hence a 
set of homogeneous linear equations with fewer equations than unknowns always 
has infinitely many solutions. 


8.18.2 N simultaneous linear equations in N unknowns 


A special case of (8.118) occurs when M = N. In this case the matrix A is square 
and we have the same number of equations as unknowns. Since A is square, the 
condition r = N corresponds to |A| # 0 and the matrix A is non-singular. The 
case r < N corresponds to |A| = 0, in which case A is singular. 

As mentioned above, the equations will have a solution provided b lies in the 
range of A. If this is true then the equations will possess a unique solution when 
|A| # 0 or infinitely many solutions when |A| = 0. There exist several methods 
for obtaining the solution(s). Perhaps the most elementary method is Gaussian 
elimination; this method is discussed in subsection 27.3.1, where we also address 
numerical subtleties such as equation interchange (pivoting). In this subsection, 
we will outline three further methods for solving a square set of simultaneous 
linear equations. 


Direct inversion 


Since A is square it will possess an inverse, provided |A| # 0. Thus, if A is 
non-singular, we immediately obtain 


x=A'b (8.122) 


as the unique solution to the set of equations. However, if b = 0 then we see 
immediately that the set of equations possesses only the trivial solution x = 0. The 
direct inversion method has the advantage that, once A~! has been calculated, 
one may obtain the solutions x corresponding to different vectors bi, bz, ... on 
the RHS, with little further work. 
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> Show that the set of simultaneous equations 
2% = 4x, SP 3x3 = 4, 
Silt 2x a 2x3 = 0, 


—3x, Sr 3x2 oP 2x3 = —7, 


has a unique solution, and find that solution. 


The simultaneous equations can be represented by the matrix equation Ax = b, ie. 


2 4 3 x1 4 
{29> 9 x» J=l o |. 
=3~ 3 2 x3 =F 


As we have already shown that A~! exists and have calculated it, see (8.59), it follows that 


x = A~'b or, more explicitly, that 
4 2 
on eee ee (8.124) 
vi 4 


xy 1 2 1 —2 
Xp =— 4 13 7 
(=) n( 3-18 a 

Thus the unique solution is x; = 2, x. = —3,x3 =4. < 
Although conceptually simple, finding the solution by calculating A~! can be 
computationally demanding, especially when N is large. In fact, as we shall now 
show, it is not necessary to perform the full inversion of A in order to solve the 
simultaneous equations Ax = b. Rather, we can perform a decomposition of the 
matrix into the product of a square lower triangular matrix L and a square upper 
triangular matrix U, which are such that 


LU decomposition 


A=LU, (8.125) 


and then use the fact that triangular systems of equations can be solved very 
simply. 

We must begin, therefore, by finding the matrices L and U such that (8.125) 
is satisfied. This may be achieved straightforwardly by writing out (8.125) in 
component form. For illustration, let us consider the 3 x 3 case. It is, in fact, 
always possible, and convenient, to take the diagonal elements of L as unity, so 
we have 


1 0 O Uy, Ui Uy 
A= Ly 1 0 0 Un, U3 
L3, L3 1 0 0 U33 
Ui Ui2 Uj3 
= [ Lay  LyUi2+Ux Lo, U43 + Ux3 (8.126) 


L3iUy,  L3U 12 + L32U22 31 U3 + L32U23 + U33 


The nine unknown elements of L and U can now be determined by equating 
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the nine elements of (8.126) to those of the 3 x 3 matrix A. This is done in the 
particular order illustrated in the example below. 

Once the matrices L and U have been determined, one can use the decomposition 
to solve the set of equations Ax = b in the following way. From (8.125), we have 
LUx = b, but this can be written as two triangular sets of equations 


Ly =b and Ux =y, 


where y is another column matrix to be determined. One may easily solve the first 
triangular set of equations for y, which is then substituted into the second set. 
The required solution x is then obtained readily from the second triangular set 
of equations. We note that, as with direct inversion, once the LU decomposition 
has been determined, one can solve for various RHS column matrices bj, bo, .. 
with little extra work. 


p> Use LU decomposition to solve the set of simultaneous equations (8.123). 


We begin the determination of the matrices L and U by equating the elements of the 
matrix in (8.126) with those of the matrix 


ca) 


2 4 3 
A= 1 -2 -2 
—3 3 2 
This is performed in the following order: 
1st row: Uy, = 2, U, = 4, Uz, = 3 
1st column: L5,U;,; = 1, L3,U 1, 3 Ly, = ; fs 3 
2nd row: Ly U12 + Ux = —2 L1U13 + Ux3 2 Ux 4, Uo; Z 
2nd column: L3,U,) + L32U» = 3 => Ly =—$ 
3rd row: L3, U3 + L32U23 + U33 = 2 => U3 = —4 
Thus we may write the matrix A as 
1 0 0 2 4 3 
A=LU= 5 1 0 0 -4 -3 
3-2 1]\o 0 -# 
We must now solve the set of equations Ly = b, which read 
1 0 O V1 4 
; 1 0 y |= 0 
oo wl Vs =] 


Since this set of equations is triangular, we quickly find 
n=4 y2=0-G)4)=-2, ys =—-7-(-3)(4) —(-)(-2) mt 


These values must then be substituted into the equations Ux = y, which read 


2 4 3 xy 4 
Oa ee x» |=] —2 
0 0 —i X3 -4 
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This set of equations is also triangular, and we easily find the solution 


x1 = 2, Xp 3, x3 =4, 


which agrees with the result found above by direct inversion. < 


We note, in passing, that one can calculate both the inverse and the determinant 
of A from its LU decomposition. To find the inverse A~!, one solves the system 
of equations Ax = b repeatedly for the N different RHS column matrices b = ¢;, 
i= 1,2,...,N, where e; is the column matrix with its ith element equal to unity 
and the others equal to zero. The solution x in each case gives the corresponding 
column of A7!. Evaluation of the determinant |A| is much simpler. From (8.125), 
we have 


|A| = |LU| = |LI|U}. (8.127) 
Since L and U are triangular, however, we see from (8.64) that their determinants 
are equal to the products of their diagonal elements. Since L;; = 1 for all i, we 
thus find 


N 
|A| = Uy,Ux2--- Unn = Il Uji. 
rar] 


As an illustration, in the above example we find |A| = (2)(—4)(—11/8) = 11, 
which, as it must, agrees with our earlier calculation (8.58). 

Finally, we note that if the matrix A is symmetric and positive semi-definite 
then we can decompose it as 


A=LL, (8.128) 


where L is a lower triangular matrix whose diagonal elements are not, in general, 
equal to unity. This is known as a Cholesky decomposition (in the special case 
where A is real, the decomposition becomes A = LL"). The reason that we cannot 
set the diagonal elements of L equal to unity in this case is that we require the 
same number of independent elements in L as in A. The requirement that the 
matrix be positive semi-definite is easily derived by considering the Hermitian 
form (or quadratic form in the real case) 


x Ax = xtLLtx = (LTx)(LTx), 


Denoting the column matrix L'x by y, we see that the last term on the RHS 
is y'y, which must be greater than or equal to zero. Thus, we require x'Ax > 0 
for any arbitrary column matrix x, and so A must be positive semi-definite (see 
section 8.17). 

We recall that the requirement that a matrix be positive semi-definite is equiv- 
alent to demanding that all the eigenvalues of A are positive or zero. If one of 
the eigenvalues of A is zero, however, then from (8.103) we have |A| = 0 and soA 
is singular. Thus, if A is a non-singular matrix, it must be positive definite (rather 
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than just positive semi-definite) in order to perform the Cholesky decomposition 
(8.128). In fact, in this case, the inability to find a matrix L that satisfies (8.128) 
implies that A cannot be positive definite. 

The Cholesky decomposition can be applied in an analogous way to the LU 
decomposition discussed above, but we shall not explore it further. 


Cramer's rule 
An alternative method of solution is to use Cramer’s rule, which also provides 
some insight into the nature of the solutions in the various cases. To illustrate 
this method let us consider a set of three equations in three unknowns, 
Aix + A1rx2 + A13x3 = br, 
Anix1 + Anx2 + Ar3x3 = bo, (8.129) 
A31X1 + Az2X2 + A33X3 = 53, 


which may be represented by the matrix equation Ax = b. We wish either to find 
the solution(s) x to these equations or to establish that there are no solutions. 
From result (vi) of subsection 8.9.1, the determinant |A| is unchanged by adding 
to its first column the combination 


= x (second column of |A|) + By (third column of |A)). 
1 x1 


We thus obtain 


Ay Aj2 Aj Att + (X2/x1)A12 + (%3/x1)A13, Ain At 
IA] =| Aa Azo Ar3 | =| Aoi + (X2/%1)A22 + (X3/%1)A23 Ar2 A23 |, 
A31 A32 A33 A31 + (X2/x1)A32 + (%3/x1)A33 32 A333 
which, on substituting b;/x; for the ith entry in the first column, yields 
1 by Ay. Ay 1 
|A| = oa by Ax. A23 | = —Aj. 


x 
‘| bs Ag. A33 : 


The determinant A; is known as a Cramer determinant. Similar manipulations of 
the second and third columns of |A| yield x2 and x3, and so the full set of results 
reads 


Ay Ad A3 
i=, MSH, B=, (8.130) 
|A| |A| |A| 
where 
by Ar Aj3 Ayn 5b Aj Au Aj by 
Ai =| by Ax. Azz |, Ar=|] Aor bz Azz}, A3 =] Art Ar do 
b3 A32_ A33 Az, 53 A33 Az, A32_ b3 


It can be seen that each Cramer determinant A; is simply |A| but with column i 
replaced by the RHS of the original set of equations. If |A| # 0 then (8.130) gives 
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the unique solution. The proof given here appears to fail if any of the solutions 
x; is zero, but it can be shown that result (8.130) is valid even in such a case. 


> Use Cramer’s rule to solve the set of simultaneous equations (8.123). 


Let us again represent these simultaneous equations by the matrix equation Ax = b, ie. 


2 4 3 x 4 
te 29, 9 x» )=l o |. 
23° 3- © x 27 


From (8.58), the determinant of A is given by |A| = 11. Following the discussion given 
above, the three Cramer determinants are 


4 4 3 2 4 3 Qa he “A 
Ape) 0°. 222-2. Age | oT 0h 2.) Ae! D> 2. 0 
=7>. 3) =). 238 3h 27 


These may be evaluated using the properties of determinants listed in subsection 8.9.1 
and we find A; = 22, A, = —33 and A; = 44. From (8.130) the solution to the equations 
(8.123) is given by 


2 253 44 
fs. Oo ag By eg 


which agrees with the solution found in the previous example. < 


X1 4, 


At this point it is useful to consider each of the three equations (8.129) as rep- 
resenting a plane in three-dimensional Cartesian coordinates. Using result (7.42) 
of chapter 7, the sets of components of the vectors normal to the planes are 
(Aq1, Ato, A13), (Ao1, A22, A23) and (A31, A32, 433), and using (7.46) the perpendic- 
ular distances of the planes from the origin are given by 

dj = — for i= 1,2,3. 
(Aj, + 4p + Ais) 
Finding the solution(s) to the simultaneous equations above corresponds to finding 
the point(s) of intersection of the planes. 

If there is a unique solution the planes intersect at only a single point. This 
happens if their normals are linearly independent vectors. Since the rows of A 
represent the directions of these normals, this requirement is equivalent to |A| # 0. 
Ifb=(0 0 0)' =0 then all the planes pass through the origin and, since there 
is only a single solution to the equations, the origin is that solution. 

Let us now turn to the cases where |A| = 0. The simplest such case is that in 
which all three planes are parallel; this implies that the normals are all parallel 
and so A is of rank 1. Two possibilities exist: 


(i) the planes are coincident, ie. dj = dy = d3, in which case there is an 
infinity of solutions; 

(ii) the planes are not all coincident, ie. d, # dy and/or d, # d3 and/or 
dy # d3, in which case there are no solutions. 
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(a) (b) 


Figure 8.1 The two possible cases when A is of rank 2. In both cases all the 
normals lie in a horizontal plane but in (a) the planes all intersect on a single 
line (corresponding to an infinite number of solutions) whilst in (b) there are 
no common intersection points (no solutions). 


It is apparent from (8.130) that case (i) occurs when all the Cramer determinants 
are zero and case (ii) occurs when at least one Cramer determinant is non-zero. 

The most complicated cases with |A| = 0 are those in which the normals to the 
planes themselves lie in a plane but are not parallel. In this case A has rank 2. 
Again two possibilities exist and these are shown in figure 8.1. Just as in the 
rank-1 case, if all the Cramer determinants are zero then we get an infinity of 
solutions (this time on a line). Of course, in the special case in which b = 0 (and 
the system of equations is homogeneous), the planes all pass through the origin 
and so they must intersect on a line through it. If at least one of the Cramer 
determinants is non-zero, we get no solution. 


These rules may be summarised as follows. 


(i) |A 
(ii) |A 
(iii) |A 
or 


(iv) |A 
(v) |A 


#0, b #0: The three planes intersect at a single point that is not the 


origin, and so there is only one solution, given by both (8.122) and (8.130). 


# 0, b = 0: The three planes intersect at the origin only and there is 


only the trivial solution, x = 0. 


= 0, b # 0, Cramer determinants all zero: There is an infinity of 


solutions either on a line if A is rank 2, i.e. the cofactors are not all zero, 


on a plane if A is rank 1, ie. the cofactors are all zero. 
= 0, b £0, Cramer determinants not all zero: No solutions. 
= 0, b = 0: The three planes intersect on a line through the origin 


giving an infinity of solutions. 


8.18.3 Singular value decomposition 


There exists a very powerful technique for dealing with a simultaneous set of 
linear equations Ax = b, such as (8.118), which may be applied whether or not 
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the number of simultaneous equations M is equal to the number of unknowns N. 
This technique is known as singular value decomposition (SVD) and is the method 
of choice in analysing any set of simultaneous linear equations. 

We will consider the general case, in which A is an M x N (complex) matrix. 
Let us suppose we can write A as the product’ 


A=usv", (8.131) 
where the matrices U, S and V have the following properties. 


(i) The square matrix U has dimensions M x M and 1s unitary. 

(ii) The matrix S has dimensions M x N (the same dimensions as those of A) 
and is diagonal in the sense that S;; = 0 if i # j. We denote its diagonal 
elements by s; for i= 1,2,...,p, where p = min(M, N); these elements are 
termed the singular values of A. 

(iii) The square matrix V has dimensions N x N and is unitary. 


We must now determine the elements of these matrices in terms of the elements of 
A. From the matrix A, we can construct two square matrices: A'A with dimensions 
NN and AA with dimensions M x M. Both are clearly Hermitian. From (8.131), 
and using the fact that U and V are unitary, we find 


Ata = vstutusvi = vsisvt (8.132) 
AAt = usvivstut = usstu, (8.133) 


where S'S and SS! are diagonal matrices with dimensions N x N and M x M 
respectively. The first p elements of each diagonal matrix are s?, PS 152s a0 Ds 
where p = min(M,N), and the rest (where they exist) are zero. 

These two equations imply that both V-'ATAV (= V-IATA(V')-!) and, by 
a similar argument, U~'AATU, must be diagonal. From our discussion of the 
diagonalisation of Hermitian matrices in section 8.16, we see that the columns of 
V must therefore be the normalised eigenvectors v'‘, i = 1,2,...,N, of the matrix 
A‘A and the columns of U must be the normalised eigenvectors u/, j = 1,2,...,M, 
of the matrix AA*. Moreover, the singular values s; must satisfy s? = /;, where 
the 4; are the eigenvalues of the smaller of ATA and AA‘. Clearly, the 4; are 
also some of the eigenvalues of the larger of these two matrices, the remaining 
ones being equal to zero. Since each matrix is Hermitian, the 1; are real and the 
singular values s; may be taken as real and non-negative. Finally, to make the 
decomposition (8.131) unique, it is customary to arrange the singular values in 
decreasing order of their values, so that sj > s7 >--- > Sp. 


8 The proof that such a decomposition always exists is beyond the scope of this book. For a 
full account of SVD one might consult, for example, G. H. Golub and C. F. Van Loan, Matrix 
Computations, 3rd edn (Baltimore MD: Johns Hopkins University Press, 1996). 
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> Show that, for i= 1,2,...,p, Av' = s;u' and A‘u! = s,v', where p = min(M, N). 


Post-multiplying both sides of (8.131) by V, and using the fact that V is unitary, we obtain 
AV = US. 


Since the columns of V and U consist of the vectors v' and u/ respectively and S has only 
diagonal non-zero elements, we find immediately that, for i = 1,2,...,p, 


Avi = syul, (8.134) 


Moreover, we note that Av' = 0 fori=p+1,p+2,...,N. 

Taking the Hermitian conjugate of both sides of (8.131) and post-multiplying by U, we 
obtain 

ATU = Vs! = VST, 
where we have used the fact that U is unitary and S is real. We then see immediately that, 
for i= 1,2,...,p, 
Atul =s,v'. (8.135) 

We also note that A‘u' = 0 for i= p+1,p+2,...,M. Results (8.134) and (8.135) are useful 
for investigating the properties of the SVD. < 


The decomposition (8.131) has some advantageous features for the analysis of 
sets of simultaneous linear equations. These are best illustrated by writing the 
decomposition (8.131) in terms of the vectors u! and v’ as 


P 
A= DS sjui(v')*, 
i=1 


where p = min(M,N). It may be, however, that some of the singular values s; 
are zero, as a result of degeneracies in the set of M linear equations Ax = b. 
Let us suppose that there are r non-zero singular values. Since our convention is 
to arrange the singular values in order of decreasing size, the non-zero singular 
values are s;, i = 1,2,...,r, and the zero singular values are s,+1,5,+2,...,Sp. 
Therefore we can write A as 


: 
A=)_ suit (8.136) 
i=l 
Let us consider the action of (8.136) on an arbitrary vector x. This is given by 
a 
Ax = 1% sjui(v')tx. 
i=l 


Since (v')*x is just a number, we see immediately that the vectors u’, i = 1,2,...,r, 
must span the range of the matrix A; moreover, these vectors form an orthonor- 
mal basis for the range. Further, since this subspace is r-dimensional, we have 
rank A =r, Le. the rank of A is equal to the number of non-zero singular values. 

The SVD is also useful in characterising the null space of A. From (8.119), 
we already know that the null space must have dimension N — r; so, if A has r 
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non-zero singular values s;, i = 1,2,...,r, then from the worked example above 
we have 


Avi =0 fori=r+1,r+2,...,N. 


Thus, the N —r vectors v', i =r+1,r+2,...,N, form an orthonormal basis for 
the null space of A. 


> Find the singular value decompostion of the matrix 


a 2D 2 2 


A= 10 10 10 ™m || < (8.137) 


The matrix A has dimension 3 x 4 (ie. M = 3, N = 4), and so we may construct from 
it the 3 x 3 matrix AA‘ and the 4 x 4 matrix A‘A (in fact, since A is real, the Hermitian 
conjugates are just transposes). We begin by finding the eigenvalues /; and eigenvectors u! 
of the smaller matrix AA‘. This matrix is easily found to be given by 


16 0 O 
aat=( 0 2 3], 
0 2s 
and its characteristic equation reads 
16-2 0 0 
0 2_) 2 | =(16—1)(36 — 134+. 27) =0. 
0 2 62g 
3 5 


Thus, the eigenvalues are 2; = 16, 2, = 9, A; = 4. Since the singular values of A are given 
by s; = /Z; and the matrix S in (8.131) has the same dimensions as A, we have 


400 0 
s={[ 0300], (8.138) 
002 0 


where we have arranged the singular values in order of decreasing size. Now the matrix U 
has as its columns the normalised eigenvectors u! of the 3 x3 matrix AA‘. These normalised 
eigenvectors correspond to the eigenvalues of AA’ as follows: 


2,=16 => w=(1 0 07 
y=9 > w=(0 3 4) 
A3=4 u =(0 2 3), 


and so we obtain the matrix 


10 0 
US| Os 2 (8.139) 
OW et 8 


The columns of the matrix V in (8.131) are the normalised eigenvectors of the 4 x 4 
matrix A'A, which is given by 


2 2 03°¢« 
if 34 29diy 3 

TA 
AR og aL “38! 
i1 3 21 29 
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We already know from the above discussion, however, that the non-zero eigenvalues of 
this matrix are equal to those of AA‘ found above, and that the remaining eigenvalue is 
zero. The corresponding normalised eigenvectors are easily found: 


A=16 = v=1i 11 1) 


a=9 = V=z1 1 -1 -1) 

23 =4 vi=t(-1 1 1 —-1)' 

da =0 vi = 5(1 11 1)" 

and so the matrix V is given by 
1 1 -1 1 
1/1 1 4 -t 
oak eee a | (8.140) 

1 -1 -1 -1 


Alternatively, we could have found the first three columns of V by using the relation 
(8.135) to obtain 
: 1 ; 
vi = —Atu! for i = 1,2,3. 
Si 
The fourth eigenvector could then be found using the Gram—Schmidt orthogonalisation 
procedure. We note that if there were more than one eigenvector corresponding to a zero 
eigenvalue then we would need to use this procedure to orthogonalise these eigenvectors 
before constructing the matrix V. 
Collecting our results together, we find the SVD of the matrix A: 


NI 
NIK 


1 1 
es 40 0 0 Pe ie nc set 
A=usvi=] 0 3 -3 0 3 0 0 an ee ee a 
0 4 3 002 0 J 3 2 73 
5 5 1 1 1 1 
Qe iD oo “So, 


this can be verified by direct multiplication. < 


Let us now consider the use of SVD in solving a set of M simultaneous linear 
equations in N unknowns, which we write again as Ax = b. Firstly, consider 
the solution of a homogeneous set of equations, for which b = 0. As mentioned 
previously, if A is square and non-singular (and so possesses no zero singular 
values) then the equations have the unique trivial solution x = 0. Otherwise, any 
of the vectors v', i =r+1,r+2,...,N, or any linear combination of them, will 
be a solution. 

In the inhomogeneous case, where b is not a zero vector, the set of equations 
will possess solutions if b lies in the range of A. To investigate these solutions, it 
is convenient to introduce the N x M matrix S, which is constructed by taking 
the transpose of S in (8.131) and replacing each non-zero singular value s; on 
the diagonal by 1/s;. It is clear that, with this construction, SS is an M x M 
diagonal matrix with diagonal entries that equal unity for those values of j for 
which s; # 0, and zero otherwise. 

Now consider the vector 


% = VSU'b. (8.141) 
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Using the unitarity of the matrices U and V, we find that 
AX — b = USSU!b — b = U(SS — I)Utb. (8.142) 


The matrix (SS — |) is diagonal and the jth element on its leading diagonal is 
non-zero (and equal to —1) only when s; = 0. However, the jth element of the 
vector U'b is given by the scalar product (u/)'b; if b lies in the range of A, this 
scalar product can be non-zero only if s; # 0. Thus the RHS of (8.142) must 
equal zero, and so X given by (8.141) is a solution to the equations Ax = b. We 
may, however, add to this solution any linear combination of the N —r vectors v’, 
i=r+1,r+2,...,N, that form an orthonormal basis for the null space of A; thus, 
in general, there exists an infinity of solutions (although it is straightforward to 
show that (8.141) is the solution vector of shortest length). The only way in which 
the solution (8.141) can be unique is if the rank r equals N, so that the matrix A 
does not possess a null space; this only occurs if A is square and non-singular. 

If b does not lie in the range of A then the set of equations Ax = b does 
not have a solution. Nevertheless, the vector (8.141) provides the closest possible 
‘solution’ in a least-squares sense. In other words, although the vector (8.141) 
does not exactly solve Ax = b, it is the vector that minimises the residual 


e = |Ax—bI, 


where here the vertical lines denote the absolute value of the quantity they 
contain, not the determinant. This is proved as follows. 

Suppose we were to add some arbitrary vector x’ to the vector x in (8.141). 
This would result in the addition of the vector b’ = Ax’ to Ak—b; b’ is clearly in 
the range of A since any part of x’ belonging to the null space of A contributes 
nothing to Ax’. We would then have 


|AX —b + b| = |(USSUT — |)b +b’ 
= |U[(SS — NU*b + Utb’]| 
= |(SS — I)Utb + U'b'; (8.143) 


in the last line we have made use of the fact that the length of a vector is left 
unchanged by the action of the unitary matrix U. Now, the jth component of the 
vector (SS — I)U*b will only be non-zero when s; = 0. However, the jth element 
of the vector U'b’ is given by the scalar product (u/)'b’, which is non-zero only if 
s; #0, since b’ lies in the range of A. Thus, as these two terms only contribute to 
(8.143) for two disjoint sets of j-values, its minimum value, as x’ is varied, occurs 
when b’ = 0; this requires x’ = 0. 


> Find the solution(s) to the set of simultaneous linear equations Ax = b, where A is given 
by (8.137) andb=(1 0 O)!. 


To solve the set of equations, we begin by calculating the vector given in (8.141), 
x = VSU'b, 
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where U and V are given by (8.139) and (8.140) respectively and S is obtained by taking 
the transpose of S in (8.138) and replacing all the non-zero singular values s; by 1/s;. Thus, 
S reads 


oO OFF 
oor oO 
oo 


Oo 
Onk 


x=i(1 1 1 0. (8.144) 


It is straightforward to show that this solves the set of equations Ax = b exactly, and 
so the vector b = (1 0 0)! must lie in the range of A. This is, in fact, immediately 
clear, since b = u!. The solution (8.144) is not, however, unique. There are three non-zero 
singular values, but N = 4. Thus, the matrix A has a one-dimensional null space, which 
is ‘spanned’ by v‘, the fourth column of V, given in (8.140). The solutions to our set of 
equations, consisting of the sum of the exact solution and any vector in the null space of 
A, therefore lie along the line 


x=4(1 1 1 1'4oa(l -1 1 —1)', 


where the parameter « can take any real value. We note that (8.144) is the point on this 
line that is closest to the origin. < 


8.19 Exercises 


8.1 Which of the following statements about linear vector spaces are true? Where a 
statement is false, give a counter-example to demonstrate this. 


(a) Non-singular N x N matrices form a vector space of dimension N?. 

(b) Singular N x N matrices form a vector space of dimension N°. 

(c) Complex numbers form a vector space of dimension 2. 

(d) Polynomial functions of x form an infinite-dimensional vector space. 

(e) Series {ao,a1,d2,...,an} for which Ng lanl? = | form an N-dimensional 
vector space. 

(f) Absolutely convergent series form an infinite-dimensional vector space. 


(g) Convergent series with terms of alternating sign form an infinite-dimensional 
vector space. 


8.2 Evaluate the determinants 
1 2 3 
ah g 
0 1 -—2 #1 
(a) h b f |, (b) 3 13 4 2 
BMG 2 eae 
and 
ge ge a+ge gb+ge 
( 0 +b b b 
c) coe e b+e 
a b b+f b+d 
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8.3 


8.4 


8.5 


8.6 


8.7 


8.8 


Using the properties of determinants, solve with a minimum of calculation the 
following equations for x: 


: : ; f x+2 x+4 x-3 
(a) Ache seta =0, (b) | x+3 x x+5|=0. 
on eee x—-2 x-1 x+l1 


Consider the matrices 


0 - i tof Ae ee, Sas 
a) B={ i 0 - ], ()c==/ 1 Ye -1 
om VB\ 2 % 2 


Are they (i) real, (ii) diagonal, (iii) symmetric, (iv) antisymmetric, (v) singular, 
(vi) orthogonal, (vii) Hermitian, (viii) anti-Hermitian, (ix) unitary, (x) normal? 
By considering the matrices 


1 0 0 0 
Asta) Pals.) 


show that AB = 0 does not imply that either A or B is the zero matrix, but that 
it does imply that at least one of them is singular. 

This exercise considers a crystal whose unit cell has base vectors that are not 
necessarily mutually orthogonal. 


(a) The basis vectors of the unit cell of a crystal, with the origin O at one corner, 
are denoted by ej, e2, e3. The matrix G has elements G;j, where Gij = e; - e; 
and Hj; are the elements of the matrix H = G7!. Show that the vectors 
fp= yy) Hye; are the reciprocal vectors and that Hj; = f;- f;. 

(b) If the vectors u and v are given by 


u= y Ue, v= y vf;, 
i i 


obtain expressions for |ul, |v], and u- v. 

(c) If the basis vectors are each of length a and the angle between each pair is 
m/3, write down G and hence obtain H. 

(d) Calculate (i) the length of the normal from O onto the plane containing the 
points p—'e;, q~'e:, r~'e3, and (ii) the angle between this normal and e. 


Prove the following results involving Hermitian matrices: 


(a) If A is Hermitian and U is unitary then U~'AU is Hermitian. 

(b) If A is anti-Hermitian then iA is Hermitian. 

(c) The product of two Hermitian matrices A and B is Hermitian if and only if 
A and B commute. 

(d) If S is a real antisymmetric matrix then A = (I— $)(1+ S)~! is orthogonal. 


If A is given by 
A= cosO sind 
~ \ =sin@ cosé 


then find the matrix S that is needed to express A in the above form. 
(e) If K is skew-hermitian, ie. K' = —K, then V = (I+ K)(l— K)~! is unitary. 
A and B are real non-zero 3 x 3 matrices and satisfy the equation 
(AB)' + B-'A=0. 


(a) Prove that if B is orthogonal then A is antisymmetric. 
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8.9 


(b) Without assuming that B is orthogonal, prove that A is singular. 
The commutator [X,Y] of two matrices is defined by the equation 

[X, Y] = XY — YX. 
Two anticommuting matrices A and B satisfy 


A? =|, B* =|, [A, B] = 2iC. 


(a) Prove that C? =| and that [B,C] = 2iA. 
(b) Evaluate [[[A, B], [B, C]], [A, B]]. 


The four matrices S,, S,, S, and | are defined by 


01 0 =i 
a(t) s(t o) 
1 0 1 0 
solo“) (ot) 


where i? = —1. Show that Ss = land S,S, = iS,, and obtain similar results 
by permutting x, y and z. Given that v is a vector with Cartesian components 
(vx, Uy, U;), the matrix S(y) is defined as 


S(v) = 0xSx + vySy +0,S,. 
Prove that, for general non-zero vectors a and b, 
S(a)S(b) = a- b1+iS(a x b). 


Without further calculation, deduce that S(a) and S(b) commute if and only if a 
and b are parallel vectors. 


A general triangle has angles x, B and y and corresponding opposite sides a, 
b and c. Express the length of each side in terms of the lengths of the other 
two sides and the relevant cosines, writing the relationships in matrix and vector 
form, using the vectors having components a, b,c and cosa, cos f, cosy. Invert the 
matrix and hence deduce the cosine-law expressions involving «, B and y. 


Given a matrix 
1 0 
A=| 8B 0 |}, 
0 1 


OrR 


where « and f are non-zero complex numbers, find its eigenvalues and eigenvec- 
tors. Find the respective conditions for (a) the eigenvalues to be real and (b) the 
eigenvectors to be orthogonal. Show that the conditions are jointly satisfied if 
and only if A is Hermitian. 


Using the Gram-—Schmidt procedure: 


(a) construct an orthonormal set of vectors from the following: 


x=(0 0 1 1, w» = 0 —1 OF, 
~=(1 2 0 2) x= 1 1 JD? 
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(b) find an orthonormal basis, within a four-dimensional Euclidean space, for 
the subspace spanned by the three vectors (1 2 0 O)',(3 —1 2 0) 
and(O 0 2 1)f. 


If a unitary matrix U is written as A+ iB, where A and B are Hermitian with 


non-degenerate eigenvalues, show the following: 


a) A and B commute; 
) A+B? =I; 

c) The eigenvectors of A are also eigenvectors of B; 

d) The eigenvalues of U have unit modulus (as is necessary for any unitary 
matrix). 


Determine which of the matrices below are mutually commuting, and, for those 
that are, demonstrate that they have a complete set of eigenvectors in common: 


6 —2 1 8 
a=( $ ra B= ( § ee 
-9 -10 14 2 
e=( “i 5 o=( 3 fi) 
Find the eigenvalues and a set of eigenvectors of the matrix 
1 3. -1 
3 4 -2 
-1 —2 2 


Verify that its eigenvectors are mutually orthogonal. 
Find three real orthogonal column matrices, each of which is a simultaneous 
eigenvector of 


00 1 0 1 
A= 0 1 0 and B= 1 0 
100 i eal 


Orr 


Use the results of the first worked example in section 8.14 to evaluate, without 
repeated matrix multiplication, the expression A°x, where x =(2 4 —1)? and 
A is the matrix given in the example. 

Given that A is a real symmetric matrix with normalised eigenvectors e', obtain 
the coefficients «; involved when column matrix x, which is the solution of 


AX — Ux =v, (*) 


is expanded as x = )°, ae’. Here y is a given constant and v is a given column 
matrix. 


(a) Solve (*) when 


w=2andv=(1 2 3)!. 
(b) Would (*) have a solution if w = 1 and (i) v = (1 2. 3)’, (ii) v = 
(2 2 3)'? 
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8.20 


8.21 


8.22 


8.23 


8.24 


8.25 


Demonstrate that the matrix 


2 0 0 
A=[{ 6 4 4 
3-1 0 


is defective, ie. does not have three linearly independent eigenvectors, by showing 

the following: 

(a) its eigenvalues are degenerate and, in fact, all equal; 

(b) any eigenvector has the form (u (3u—2v) v)!. 

(c) if two pairs of values, p14, ¥; and po, v2, define two independent eigenvectors 
v, and v2, then any third similarly defined eigenvector v3 can be written as 
a linear combination of v; and vo, ie. 


V3 = av, + bva, 
where 
qs H3V2 — P2V3 and. p= HiV3 TBM 
Miv2 — P2v1 Hiv2 — Pov 
Illustrate (c) using the example (t1,v,) = (1,1), (40,2) = (1,2) and (43,3) = 
(0, 1). 
Show further that any matrix of the form 
2 0 0 
6n—6 4—2n 4—4n 
3-—3n n-1 2n 


is defective, with the same eigenvalues and eigenvectors as A. 
By finding the eigenvectors of the Hermitian matrix 


103i 
n= (35 2 i 


construct a unitary matrix U such that U'HU = A, where A is a real diagonal 
matrix. 
Use the stationary properties of quadratic forms to determine the maximum and 
minimum values taken by the expression 

QO = 5x? + 4y? + 427 4 2xz + 2xy 


on the unit sphere, x? + y?+z? = 1. For what values of x, y and z do they occur? 
Given that the matrix 


has two eigenvectors of the form (1 y 1)", use the stationary property of the 
expression J(x) = x'Ax/(x'x) to obtain the corresponding eigenvalues. Deduce 
the third eigenvalue. 

Find the lengths of the semi-axes of the ellipse 


73x" + 72xy + 52y* = 100, 


and determine its orientation. 
The equation of a particular conic section is 


O = 8x7 + 8x5 — 6x1 x2 = 110. 


Determine the type of conic section this represents, the orientation of its principal 
axes, and relevant lengths in the directions of these axes. 
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8.26 


8.28 


8.29 


Show that the quadratic surface 
5x? + l1y? + 52? — 10yz + 2xz — 10xy = 4 


is an ellipsoid with semi-axes of lengths 2, 1 and 0.5. Find the direction of its 
longest axis. 
Find the direction of the axis of symmetry of the quadratic surface 


Tx? + Ty? + 72? — 20yz — 20xz + 20xy = 3. 


For the following matrices, find the eigenvalues and sufficient of the eigenvectors 
to be able to describe the quadratic surfaces associated with them: 


ae | 1.922 ae ae 
(a) { 1 5 1 |, mf 212), Mf 2 4 2 
mi a ee os i 1204 


This exercise demonstrates the reverse of the usual procedure of diagonalising a 
matrix. 


(a) Rearrange the result A’ = S~'AS of section 8.16 to express the original 
matrix A in terms of the unitary matrix S and the diagonal matrix A’. Hence 
show how to construct a matrix A that has given eigenvalues and given 
(orthogonal) column matrices as its eigenvectors. 

(b) Find the matrix that has as eigenvectors (1 2 1)', (1 —1 1)" and 
(1 0 —1)', with corresponding eigenvalues 2, u and v. 

(c) Try a particular case, say 2 = 3, w 2 and v = 1, and verify by explicit 
solution that the matrix so found does have these eigenvalues. 


Find an orthogonal transformation that takes the quadratic form 
= =x? _ 2x} _ x} + 8x2x3 + 6x1x3 + 8X1 x2 
into the form 
my? + poy; — 4y3, 
and determine pu; and fp (see section 8.17). 


One method of determining the nullity (and hence the rank) of an M x N matrix 
A is as follows. 


e Write down an augmented transpose of A, by adding on the right an N x N 
unit matrix and thus producing an N x (M+ N) array B. 

e Subtract a suitable multiple of the first row of B from each of the other lower 
rows so as to make B,, = 0 for i> 1. 

e Subtract a suitable multiple of the second row (or the uppermost row that 
does not start with M zero values) from each of the other lower rows so as to 
make B,. = 0 for i > 2. 

e Continue in this way until all remaining rows have zeros in the first M places. 
The number of such rows is equal to the nullity of A, and the N rightmost 
entries of these rows are the components of vectors that span the null space. 
They can be made orthogonal if they are not so already. 


Use this method to show that the nullity of 


-1 3 2 7 
3. 10 -6 17 
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8.32 


8.33 


8.37 


8.38 


is 2 and that an orthogonal base for the null space of A is provided by any two 
column matrices of the form (2+; —20; 1 «;)', for which the a; (i = 1,2) 
are real and satisfy 60,02 + 2(%, + %)+5=0. 

Do the following sets of equations have non-zero solutions? If so, find them. 


(a) 3x+2y+z=0, x—3y+2z=0, 2x+y+3z=0. 
(b) 2x = b(y +2), x = 2a(y —2z), x = (6a—b)y — (6a+ b)z. 
Solve the simultaneous equations 
2x+3y+z=11, 
x+y+z=6, 
5x— y+ 10z = 34. 


Solve the following simultaneous equations for x;, x. and x3, using matrix 
methods: 


Xi vr 2x2 + 3x3 = 1, 
3x1 a 4x + 5x3 = 2) 
xXitr 3x2 + 4x3 S33. 


Show that the following equations have solutions only if 7 = 1 or 2, and find 
them in these cases: 


x+ytz=1, 
x+2y+4z =n, 
x+4y +102 = 7’. 


Find the condition(s) on « such that the simultaneous equations 


x, tax. =1, 
Xi X24 3x3 = -1, 
2x1 — 2x7 + ax3 = —2 


have (a) exactly one solution, (b) no solutions, or (c) an infinite number of 
solutions; give all solutions where they exist. 
Make an LU decomposition of the matrix 


3 6 9 
A=j;1 0 5 
2 —2 16 


and hence solve Ax = b, where (i) b= (21 9 28)", (ii) bb=(21 7 22)'. 
Make an LU decomposition of the matrix 


2 -3 1 3 
1 4 -3 -3 
$5 3 -1 -1 
3 -6 -3 1 


Hence solve Ax = b for (i)b=(—4 1 8 —5)',(ii)b=(-10 0 —3 —24)!. 
Deduce that det A = —160 and confirm this by direct calculation. 

Use the Cholesky separation method to determine whether the following matrices 
are positive definite. For each that is, determine the corresponding lower diagonal 
matrix L: 


A= 


2-1. 3 5 0 3 
A=|{|1 3 -1], B=/[ 0 3 0 
3 -1 1 V3 0 3 
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8.40 


8.41 


8.42 


8.43 


8.1 


8.3 


Find the equation satisfied by the squares of the singular values of the matrix 
associated with the following over-determined set of equations: 


2x+3y+z=0 

x-—y-z=l1 

2x+y=0 
2y+z=—2. 


Show that one of the singular values is close to zero. Determine the two larger 
singular values by an appropriate iteration process and the smallest one by 
indirect calculation. 

Find the SVD of 


showing that the singular values are 3 and 1. 
Find the SVD form of the matrix 


22 28 —-2 
1 2 -19 
Bg ed 
Se oS 6 


Use it to determine the best solution x of the equation Ax = b when (i) b = 
(6 —39 15 18)", (iij)b=(9 —42 15 15)", showing that (i) has an exact 
solution, but that the best solution to (ii) has a residual of 4/78. 
Four experimental measurements of particular combinations of three physical 
variables, x, y and z, gave the following inconsistent results: 
13x + 22y — 13z =4, 
10x — 8y — 10z = 44, 
10x — 8y — 10z = 47, 
9x — 18y —9z = 72. 

Find the SVD best values for x, y and z. Identify the null space of A and hence 
obtain the general SVD solution. 


8.20 Hints and answers 


(a) False. Oy, the N x N null matrix, is not non-singular. 

(b) False. Consider the sum of ( , ) and ( ; ). 

(c) True. 

(d) True. 

(e) False. Consider b, = a, +a, for which soe |b, (> = 4 # 1, or note that there 
is no zero vector with unit norm. 


(f) True. 
(g) False. Consider the two series defined by 
ao = 7 a= 2(-4)" for n>1; y= —(—4)" for n>0. 


The series that is the sum of {a,} and {b,} does not have alternating signs 
and so closure does not hold. 
(a) x =a, b or c; (b) x = —1; the equation is linear in x. 
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8.5 
8.7 


8.9 
8.11 


8.13 


8.15 


8.41 


8.43 


Use the property of the determinant of a matrix product. 
0 —tan(0/2) 

(d)S= ( tan(0/2) 0 ) 
(e) Note that (I-+ K)(I— K) =1—k? = (I—K)(-+ K). 
(b) 32iA. 
a= bcosy +ccosf, and cyclic permutations; a? = b* +c? — 2becos a, and cyclic 
permutations. 
(a) 2-720 0 1 1)",622 0 —-1 1, 

30-71 6 —1 1F, 13-722 1 2 —2)7. 
(b) 5/21 2 0 0)", B45)'204 —7 10 Op, 

(18 285)-/2(-56 28 98 69)?. 
C does not commute with the others; A, B and D have (1 —2)' and (2 1)? 
common eigenvectors. 
ForA: (1 0 —1)',Q m1 DU mw 1). 
ForB: (1 1 1)7, (Bi v1 — Biv)", (B22 — Bo - 2)" 
The «;, 6; and y; are arbitrary. 
Simultaneous and orthogonal:(1 0 —1)',(2 1 DT, —2 1). 

= (v- e”)/(A; — w), where J; is the eigenvalue corresponding to e’. 


(a) x=(2 1. 3)7. 
(b) Since y is equal to one of A’s eigenvalues 4;, the equation only has a solution 
if v- e/* = 0; (i) no solution; (ii) x =(1_ 1 3/2)! 


U = (10)7!/°(1, 3i; 3i, 1), A = (1,0;0, 11). 
J = (2y’—4y4+4)/(y’ +2) ), with stationary values at y = +./2 and corresponding 
eigenvalues 2 + ./2. From the trace property of A, the third eigenvalue equals 2. 
Ellipse; 0 = 2/4, a = /22; 0 = 32/4, b = ,/10. 
The direction of the eigenvector having the unrepeated eigenvalue is 
(1-1/3. 
(a) A =SA’S', where S is the matrix whose columns are the eigenvectors of the 
matrix A to be constructed, and A’ = diag (/, 11, v). 
(b) A= (A+ 2u4 3, 24 — 2p, A+ 2u—3v; 2A— 2p, 404 2p, 21 — 2; 
A+ 2p — 3v, 2A— 2p, A+ 2" 4+ 3y). 
(c) 4+(1,5,-2;5,4,5;—2,5, 1). 
The wll space is spanned by (2 0 1 O)' and(1 —2 0 1). 
x= 32 pS 22, 
First show that A is singular. 7 = 1, x = 14+ 2z, y = —3z; n = 2, x = 2z, 


Sale Te 
L = (1,0,0; 4, er 1), U = (3,6,9;0,—2,2;0,0,4). 
()x=(-1 1 2)’. @i)x=(-3 2 2)7. 


A is not positive ae as L33 is calculated to be /—6. 
B =LL!, where the non-zero elements of L are 


Ly = V5, La = 3/5, Lx = V3, Ls3 = y/12/5. 


wa=(? 1),u-4 . YM ve4(3 i 

a Gn mea bam eK 1-1 
V6\ 1 8 Of v2 

The singular values are 12,/6,0,18./3 and the calculated best solution is x = 


1.71, y = —1.94,z = —1.71. The null space is the line x = z, y = 0 and the general 
SVD solution is x = 1.71+4/, y = —1.94, z =—-1.714+2. 
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Normal modes 


Any student of the physical sciences will encounter the subject of oscillations on 
many occasions and in a wide variety of circumstances, for example the voltage 
and current oscillations in an electric circuit, the vibrations of a mechanical 
structure and the internal motions of molecules. The matrices studied in the 
previous chapter provide a particularly simple way to approach what may appear, 
at first glance, to be difficult physical problems. 

We will consider only systems for which a position-dependent potential exists, 
i1e., the potential energy of the system in any particular configuration depends 
upon the coordinates of the configuration, which need not be be lengths, however; 
the potential must not depend upon the time derivatives (generalised velocities) of 
these coordinates. So, for example, the potential —qv- A used in the Lagrangian 
description of a charged particle in an electromagnetic field is excluded. A 
further restriction that we place is that the potential has a local minimum at 
the equilibrium point; physically, this is a necessary and sufficient condition for 
stable equilibrium. By suitably defining the origin of the potential, we may take 
its value at the equilibrium point as zero. 

We denote the coordinates chosen to describe a configuration of the system 
by qi, i = 1,2,...,N. The q; need not be distances; some could be angles, for 
example. For convenience we can define the qg; so that they are all zero at the 
equilibrium point. The instantaneous velocities of various parts of the system will 
depend upon the time derivatives of the qj, denoted by q;. For small oscillations 
the velocities will be linear in the g; and consequently the total kinetic energy T 
will be quadratic in them — and will include cross terms of the form qq; with 
i # j. The general expression for T can be written as the quadratic form 


C= = ye aij4idj = 4" AQ, (9.1) 
1 J 


where q is the column vector (4; @ --- qn)’ and the N x N matrix A 
is real and may be chosen to be symmetric. Furthermore, A, like any matrix 
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corresponding to a kinetic energy, is positive definite; that is, whatever non-zero 
real values the q; take, the quadratic form (9.1) has a value > 0. 

Turning now to the potential energy, we may write its value for a configuration 
q by means of a Taylor expansion about the origin q = 0, 


avi 4 2V(0) 
Via) =VO)+ Dai #39 aaa gia + 
qi0g; 
However, we have chosen V(0) = 0 and, since the origin is an equilibrium point, 
there is no force there and 0V(0)/0q; = 0. Consequently, to second order in the 
qi we also have a quadratic form, but in the coordinates rather than in their time 


derivatives: 
V=S—S¢ bijaiq; = 9°Ba, (9.2) 
ij 


where B is, or can be made, symmetric. In this case, and in general, the requirement 
that the potential is a minimum means that the potential matrix B, like the kinetic 
energy matrix A, is real and positive definite. 


9.1 Typical oscillatory systems 


We now introduce particular examples, although the results of this section are 
general, given the above restrictions, and the reader will find it easy to apply the 
results to many other instances. 

Consider first a uniform rod of mass M and length /, attached by a light string 
also of length | to a fixed point P and executing small oscillations in a vertical 
plane. We choose as coordinates the angles 0; and 0) shown, with exaggerated 
magnitude, in figure 9.1. In terms of these coordinates the centre of gravity of the 
rod has, to first order in the 6;, a velocity component in the x-direction equal to 
10, + 5105 and in the y-direction equal to zero. Adding in the rotational kinetic 
energy of the rod about its centre of gravity we obtain, to second order in the 6j, 


T ~ MP (6; + 403 + 0102) + HMPO3 


‘ eof : : 62-32 N34 
= ¢MP (30; + 30:02 + 63) = 4MPq" ( a8 ) 4, (9.3) 
where G' = (0, 65). The potential energy is given by 
V = Mie [(1 — cos 01) + 5(1 — cos 02)} (9.4) 
so that 
6 0 
Vz 4Mlg(20; +03) = 4Mleq! ( 0 3 ) qa, (9.5) 


where g is the acceleration due to gravity and q = (0; 0)"; (9.5) is valid to 
second order in the 0;. 
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(a) 


Figure 9.1 A uniform rod of length / attached to the fixed point P by a light 
string of the same length: (a) the general coordinate system; (b) approximation 


to the normal mode with lower frequency; (c) approximation to the mode with 
higher frequency. 


With these expressions for T and V we now apply the conservation of energy, 


“(r +V)=0, (9.6) 


assuming that there are no external forces other than gravity. In matrix form 
(9.6) becomes 


d i... Tak) aTac : 
F(a Ad + 4) Ba) = GAG + 4° AG + 4'Bq + q'Bq = 0, 
which, using A = A! and B = B/, gives 
2q1(Ag + Bq) = 0. 


We will assume, although it is not clear that this gives the only possible solution, 


that the above equation implies that the coefficient of each q; is separately zero. 
Hence 


Ag + Bq =0. (9.7) 


For a rigorous derivation Lagrange’s equations should be used, as in chapter 22. 
Now we search for sets of coordinates q that all oscillate with the same period, 


ie. the total motion repeats itself exactly after a finite interval. Solutions of this 
form will satisfy 


= xcosat; 9.8) 
q ( 


the relative values of the elements of x in such a solution will indicate how each 
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coordinate is involved in this special motion. In general there will be N values 
of w if the matrices A and B are N x N and these values are known as normal 
frequencies or eigenfrequencies. 

Putting (9.8) into (9.7) yields 


w° Ax + Bx = (B— 7A)x = 0. (9.9) 


Our work in section 8.18 showed that this can have non-trivial solutions only if 
|B—w*A| = 0. (9.10) 


This is a form of characteristic equation for B, except that the unit matrix | has 
been replaced by A. It has the more familiar form if a choice of coordinates is 
made in which the kinetic energy T is a simple sum of squared terms, i.e. it has 
been diagonalised, and the scale of the new coordinates is then chosen to make 
each diagonal element unity. 

However, even in the present case, (9.10) can be solved to yield wz for k = 
1,2,...,N, where N is the order of A and B. The values of wm, can be used 
with (9.9) to find the corresponding column vector x‘ and the initial (stationary) 
physical configuration that, on release, will execute motion with period 21/a,. 

In equation (8.76) we showed that the eigenvectors of a real symmetric matrix 
were, except in the case of degeneracy of the eigenvalues, mutually orthogonal. 
In the present situation an analogous, but not identical, result holds. It is shown 
in section 9.3 that if x! and x? are two eigenvectors satisfying (9.9) for different 
values of w* then they are orthogonal in the sense that 

(x?)TAx! = 0 and (x?)' Bx! = 0. 
The direct ‘scalar product’ (x*)'x!, formally equal to (x?)"Ix! 
equal to zero. 

Returning to the suspended rod, we find from (9.10) 


Mig/6 0\_ oMP (6 3\|_, 
12 0 3 12 3 2); 7 
Writing w71/g = 2, this becomes 


6-64 —3A 
—3A 3-—2A 


, is not, in general, 


=0 => VL-10i+6=0, 


which has roots 4 = 5+ ,/19. Thus we find that the two normal frequencies are 
given by w, = (0.641¢/1)'/? and w) = (9.359¢/1)'/*. Putting the lower of the two 
values for w?, namely (5 — J19)g/1, into (9.9) shows that for this mode 


x1 1X2 = 3(5— v/19) : 6(/19 — 4) = 1.923 : 2.153. 


This corresponds to the case where the rod and string are almost straight out, Le. 
they almost form a simple pendulum. Similarly it may be shown that the higher 
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frequency corresponds to a solution where the string and rod are moving with 
opposite phase and x; : x2 = 9.359 : —16.718. The two situations are shown in 
figure 9.1. 

In connection with quadratic forms it was shown in section 8.17 how to make 
a change of coordinates such that the matrix for a particular form becomes 
diagonal. In exercise 9.6 a method is developed for diagonalising simultaneously 
two quadratic forms (though the transformation matrix may not be orthogonal). 
If this process is carried out for A and B in a general system undergoing stable 
oscillations, the kinetic and potential energies in the new variables y; take the 
forms 


T => win? =™M, M = diag (iu, J0,..., HN), (9.11) 


ri 


V = Sovin? =n'Nn, N= diag (v1, v2...,vN), (9.12) 


ri 


and the equations of motion are the uncoupled equations 
hiiji + vini = 0, i= 1,2,...,N. (9.13) 


Clearly a simple renormalisation of the 4; can be made that reduces all the 4; 
in (9.11) to unity. When this is done the variables so formed are called normal 
coordinates and equations (9.13) the normal equations. 

When a system is executing one of these simple harmonic motions it is said to 
be in a normal mode, and once started in such a mode it will repeat its motion 
exactly after each interval of 2x/m;. Any arbitrary motion of the system may 
be written as a superposition of the normal modes, and each component mode 
will execute harmonic motion with the corresponding eigenfrequency; however, 
unless by chance the eigenfrequencies are in integer relationship, the system will 
never return to its initial configuration after any finite time interval. 

As a second example we will consider a number of masses coupled together by 
springs. For this type of situation the potential and kinetic energies are automat- 
ically quadratic functions of the coordinates and their derivatives, provided the 
elastic limits of the springs are not exceeded, and the oscillations do not have to 
be vanishingly small for the analysis to be valid. 


> Find the normal frequencies and modes of oscillation of three particles of masses m, um, 
m connected in that order in a straight line by two equal light springs of force constant k. 
This arrangement could serve as a model for some linear molecules, e.g. COd. 


The situation is shown in figure 9.2; the coordinates of the particles, x1, x2, x3, are 
measured from their equilibrium positions, at which the springs are neither extended nor 
compressed. 

The kinetic energy of the system is simply 


T = $m (x + 455 +53), 
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Figure 9.2 Three masses m, um and m connected by two equal light springs 
of force constant k. 


ee O 
(©) Onw@ ~e _EO 


Figure 9.3. The normal modes of the masses and springs of a linear molecule 
such as CO. (a) wm? = 0; (b) w? = k/m; (c) w? = [(u + 2)/u](k/m). 


whilst the potential energy stored in the springs is 
V = $k [(x2 — x1)? + 0X3 — 99)’. 


The kinetic- and potential-energy symmetric matrices are thus 


1 0 0 1 -1 0 
ASSO Ole | Mee ag Se 
001 2\ 0 -1 1 


From (9.10), to find the normal frequencies we have to solve |B — w?A| = 0. Thus, writing 
mo*/k = 2, we have 
1-1 -1 0 
1 2-ps 1 | =0, 
0 -1 1-2 


which leads to 4 = 0, 1 or 1+ 2/p. The corresponding eigenvectors are respectively 


1 1 1 
sf ak 1 |, gees 0 |, | ns —2/u 


Bhs ON ey V24+471e) \ 1 


The physical motions associated with these normal modes are illustrated in figure 9.3. 
The first, with 2 = w = 0 and all the x; equal, merely describes bodily translation of the 
whole system, with no (i.e. zero-frequency) internal oscillations. 

In the second solution the central particle remains stationary, x. = 0, whilst the other 
two oscillate with equal amplitudes in antiphase with each other. This motion, which has 
frequency w = (k/m)'/”, is illustrated in figure 9.3(b). 
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The final and most complicated of the three normal modes has angular frequency 
o = {[(u + 2)/u](k/m)}'7, and involves a motion of the central particle which is in 
antiphase with that of the two outer ones and which has an amplitude 2/ times as great. 
In this motion (see figure 9.3(c)) the two springs are compressed and extended in turn. We 
also note that in the second and third normal modes the centre of mass of the molecule 
remains stationary. < 


9.2 Symmetry and normal modes 


It will have been noticed that the system in the above example has an obvious 
symmetry under the interchange of coordinates 1 and 3: the matrices A and B, 
the equations of motion and the normal modes illustrated in figure 9.3 are all 
unaltered by the interchange of x; and —x3. This reflects the more general result 
that for each physical symmetry possessed by a system, there is at least one 
normal mode with the same symmetry. 

The general question of the relationship between the symmetries possessed by 
a physical system and those of its normal modes will be taken up more formally 
in chapter 29 where the representation theory of groups is considered. However, 
we can show here how an appreciation of a system’s symmetry properties will 
sometimes allow its normal modes to be guessed (and then verified), something 
that is particularly helpful if the number of coordinates involved is greater than 
two and the corresponding eigenvalue equation (9.10) is a cubic or higher-degree 
polynomial equation. 

Consider the problem of determining the normal modes of a system consist- 
ing of four equal masses M at the corners of a square of side 2L, each pair 
of masses being connected by a light spring of modulus k that is unstretched 
in the equilibrium situation. As shown in figure 9.4, we introduce Cartesian 
coordinates X,,y,, with n = 1,2,3,4, for the positions of the masses and de- 
note their displacements from their equilibrium positions R, by qy = Xpi + y,j. 
Thus 


r=R,t+q, with R, = +Li+ Lj. 


The coordinates for the system are thus x1,)1,X2,...,y4 and the kinetic en- 
ergy matrix A is given trivially by Mls, where lg is the 8 x 8 identity ma- 
trix. 

The potential energy matrix B is much more difficult to calculate and involves, 
for each pair of values m,n, evaluating the quadratic approximation to the 
expression 


2 
bmn = 5k (Itm Trl — [Rin R,,|) ‘ 
Expressing each r; in terms of q; and R; and making the normal assumption that 
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Figure 9.4 The arrangement of four equal masses and six equal springs 
discussed in the text. The coordinate systems x,, y, for n = 1,2,3,4 measure 
the displacements of the masses from their equilibrium positions. 


IRin = R,| > Im = qui. we obtain bin (= bym): 


rm = $k [|(Rm — Rn) + (Gm = n)| = [Rm — Rul] 
= 3k { [Rm — Rul? + 2€dm — 4a) - (Ra — Rn) + Id — @n)17] 


2(dm — dn) «(Rav — Ry) Lees 
= $kIRn — Rul? E ; t | 1 


1/2 2 
: _ IRin —R,|} 


IRn ms R,,/? 


Ag 1K (Gin = Qn) : (Ru eT R,) 
2 IRin = R,| : 


This final expression is readily interpretable as the potential energy stored in the 
spring when it is extended by an amount equal to the component, along the 
equilibrium direction of the spring, of the relative displacement of its two ends. 

Applying this result to each spring in turn gives the following expressions for 
the elements of the potential matrix. 


mn 2bin/k 
a 
2 (x1 — x2) 
(yi — 3)? 


5(—x1 +. x4 + y1 — ya)? 
5(x2 — x3 + y2 — 3)? 
(y2 — ya)? 
(x3 — x4)’. 


WNN RRR 
RW HW 
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The potential matrix is thus constructed as 


3 -1 —2 0 0 0 -1 1 
-1 3 0 O oO -2 1 -1 


Be k 0 O 3 -1 -1 0 -—2 
4 0 oO -1 -1 3 1 —2 O 

0 —2 -1 -1 1 3 0 0 

-1 1 0 —2 #O 3 -1 


1 -1 0 —2 0O oO -1 3 


To solve the eigenvalue equation |B — /A| = 0 directly would mean solving 
an eigth-degree polynomial equation. Fortunately, we can exploit intuition and 
the symmetries of the system to obtain the eigenvectors and corresponding 
eigenvalues without such labour. 

Firstly, we know that bodily translation of the whole system, without any 
internal vibration, must be possible and that there will be two independent 
solutions of this form, corresponding to translations in the x- and y- directions. 
The eigenvector for the first of these (written in row form to save space) is 


xXO—=(1 0 10 10 1 OF. 
Evaluation of Bx") gives 
BX) =(0 0 00 0 0 0 Oj, 


showing that x) is a solution of (B— w?A)x = 0 corresponding to the eigenvalue 
«w* = 0, whatever form Ax may take. Similarly, 


x7=(0 1 01010 1) 


is a second eigenvector corresponding to the eigenvalue m? = 0. 

The next intuitive solution, again involving no internal vibrations, and, there- 
fore, expected to correspond to w? = 0, is pure rotation of the whole system 
about its centre. In this mode each mass moves perpendicularly to the line joining 
its position to the centre, and so the relevant eigenvector is 


x9) = ta 11 1 11 1 1)". 
Wp 
It is easily verified that Bx?) = 0 thus confirming both the eigenvector and the 
corresponding eigenvalue. The three non-oscillatory normal modes are illustrated 
in diagrams (a)-(c) of figure 9.5. 

We now come to solutions that do involve real internal oscillations, and, 
because of the four-fold symmetry of the system, we expect one of them to be a 
mode in which all the masses move along radial lines — the so-called ‘breathing 
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a ae a a a a ae 


(a) o =0 (b) a7 =0 (c) w7? =0 (d) w? = 2k/M 
| a 


(ec) @? =k/M (f) ©? =k/M (g) w= =k/M (h) o@? =k/M 


Figure 9.5 The displacements and frequencies of the eight normal modes of 
the system shown in figure 9.4. Modes (a), (b) and (c) are not true oscillations: 
(a) and (b) are purely translational whilst (c) is a mode of bodily rotation. 
Mode (d), the ‘breathing mode’, has the highest frequency and the remaining 
four, (e)-(h), of lower frequency, are degenerate. 


mode’. Expressing this motion in coordinate form gives as the fourth eigenvector 


eae ae (2 i i St a SE, 


97) 


Evaluation of Bx) yields 


k 
Bx) = —_(-8 8 8 8 8 8 8 8)) = 2kx), 
4,/2 


ie. a multiple of x), confirming that it is indeed an eigenvector. Further, since 
Ax = Mx), it follows from (B — w*A)x = 0 that w? = 2k/M for this normal 
mode. Diagram (d) of the figure illustrates the corresponding motions of the four 
masses. 

As the next step in exploiting the symmetry properties of the system we note 
that, because of its reflection symmetry in the x-axis, the system is invariant under 
the double interchange of y; with —y3 and y) with —y4. This leads us to try an 
eigenvector of the form 


XI = « O BP 0 —a O —Ff)l. 
Substituting this trial vector into (B — w?A)x = 0 gives, of course, eight simulta- 
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neous equations for « and f, but they are all equivalent to just two, namely 


a+ B =O, 


Sa+ p= a; 


these have the solution « = —B and w* = k/M. The latter thus gives the frequency 
of the mode with eigenvector 


xI=(0 10 -1 0 -1 0 2D). 
Note that, in this mode, when the spring joining masses 1 and 3 is most stretched, 


the one joining masses 2 and 4 is at its most compressed. Similarly, based on 
reflection symmetry in the y-axis, 


x9 =(1 0 1 0 1 0 1 oj 


can be shown to be an eigenvector corresponding to the same frequency. These 
two modes are shown in diagrams (e) and (f) of figure 9.5. 

This accounts for six of the expected eight modes, and the other two could be 
found by considering motions that are symmetric about both diagonals of the 
square or are invariant under successive reflections in the x- and y- axes. However, 
since A is a multiple of the unit matrix, and since we know that (x))™Ax = 0 if 
i # j, we can find the two remaining eigenvectors more easily by requiring them 
to be orthogonal to each of those found so far. 

Let us take the next (seventh) eigenvector, x”, to be given by 


x2 =(a b oc def g hy. 


Then orthogonality with each of the x for n = 1,2,...,6 yields six equations 
satisfied by the unknowns a, b,...,h. As the reader may verify, they can be reduced 
to the six simple equations 


a+g=0, d+f=0, a+f=d+g, 
b+h=0, cte=0, b+c=e+h. 


With six homogeneous equations for eight unknowns, effectively separated into 
two groups of four, we may pick one in each group arbitrarily. Taking a = b = 1 
givesd=e=landc=f=g=h 1 as a solution. Substitution of 


xXV=(1 1 —-1 1 1 -1 -1 2). 
into the eigenvalue equation checks that it is an eigenvector and shows that the 
corresponding eigenfrequency is given by w* = k/M. 
We now have the eigenvectors for seven of the eight normal modes and the 


eighth can be found by making it simultaneously orthogonal to each of the other 
seven. It is left to the reader to show (or verify) that the final solution is 


x9 =(1 -1 1 1 -1 -1 -1 1)! 
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and that this mode has the same frequency as three of the other modes. The 
general topic of the degeneracy of normal modes is discussed in chapter 29. The 
movements associated with the final two modes are shown in diagrams (g) and 
(h) of figure 9.5; this figure summarises all eight normal modes and frequencies. 
Although this example has been lengthy to write out, we have seen that the 
actual calculations are quite simple and provide the full solution to what is 
formally a matrix eigenvalue equation involving 8 x 8 matrices. It should be 
noted that our exploitation of the intrinsic symmetries of the system played a 
crucial part in finding the correct eigenvectors for the various normal modes. 


9.3 Rayleigh—Ritz method 


We conclude this chapter with a discussion of the Rayleigh—Ritz method for 
estimating the eigenfrequencies of an oscillating system. We recall from the 
introduction to the chapter that for a system undergoing small oscillations the 
potential and kinetic energy are given by 


V=q'Bq and T=4q'AQG, 


where the components of q are the coordinates chosen to represent the configura- 
tion of the system and A and B are symmetric matrices (or may be chosen to be 
such). We also recall from (9.9) that the normal modes x’ and the eigenfrequencies 
@; are given by 


(B— w?A)x' = 0. (9.14) 


It may be shown that the eigenvectors x! corresponding to different normal modes 
are linearly independent and so form a complete set. Thus, any coordinate vector 
q can be written q = yj cx! We now consider the value of the generalised 
quadratic form 

x Bx = inl) ch BSD; cix! 

XTAX SU Q)TCEA DI, cexk? 


which, since both numerator and denominator are positive definite, is itself non- 
negative. Equation (9.14) can be used to replace Bx’, with the result that 


nx) eA So; 0? ix! 
DCAM GAD, cat 
aes Sm) cr, 1s apciAx! (9 15) 
OA TEA DY, cox ; 
Now the eigenvectors x' obtained by solving (B — w*A)x = 0 are not mutually 
orthogonal unless either A or B is a multiple of the unit matrix. However, it may 


A(x) = 


Ax) = 
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be shown that they do possess the desirable properties 
(x/)TAx'=0 and (x/)’Bx' =0 ifi#j. (9.16) 
This result is proved as follows. From (9.14) it is clear that, for general i and j, 
(x/)"(B — 7 A)x' = 0. (9.17) 


But, by taking the transpose of (9.14) with i replaced by j and recalling that A 
and B are real and symmetric, we obtain 


(x/)"(B— 7A) = 0. 


Forming the scalar product of this with x' and subtracting the result from (9.17) 
gives 
(0; _ oo? )(x/)T Ax’ =0. 
Thus, for i # j and non-degenerate eigenvalues w? and ;, we have that 
(x/)TAx! = 0, and substituting this into (9.17) immediately establishes the corre- 
sponding result for (x/)'Bx'. Clearly, if either A or B is a multiple of the unit 
matrix then the eigenvectors are mutually orthogonal in the normal sense. The 
orthogonality relations (9.16) are derived again, and extended, in exercise 9.6. 
Using the first of the relationships (9.16) to simplify (9.15), we find that 


=> ei/?0?(x)TAx! 


MX) = SS To TARE” 


(9.18) 


Now, if «4 is the lowest eigenfrequency then w? > «4 for all i and, further, since 
(x‘)TAx' > 0 for all i the numerator of (9.18) is > w@ 7; |cil?(x‘)TAx'. Hence 


> w, (9.19) 


for any x whatsoever (whether x is an eigenvector or not). Thus we are able to 
estimate the lowest eigenfrequency of the system by evaluating 4 for a variety 
of vectors x, the components of which, it will be recalled, give the ratios of the 
coordinate amplitudes. This is sometimes a useful approach if many coordinates 
are involved and direct solution for the eigenvalues is not possible. 

An additional result is that the maximum eigenfrequency w2, may also be 
estimated. It is obvious that if we replace the statement ‘w? > « for all ? by 
‘w? < wz, for all 7, then A(x) < wz, for any x. Thus A(x) always lies between 
the lowest and highest eigenfrequencies of the system. Furthermore, /(x) has a 
stationary value, equal to wz, when x is the kth eigenvector (see subsection 8.17.1). 
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> Estimate the eigenfrequencies of the oscillating rod of section 9.1. 


Firstly we recall that 


MP / 6 3 Mig (6 0 
a-"5(5 2) and a= 5 3 


Physical intuition suggests that the slower mode will have a configuration approximating 
that of a simple pendulum (figure 9.1), in which 0, = 02, and so we use this as a trial 
vector. Taking x = (0 6)", 


T 2 
x'Bx 3MIg6*/4 9g 0.6438, 


400 = Sax ~ 7MP02/6 ~ T4l i 


and we conclude from (9.19) that the lower (angular) frequency is < (0.643g/I)!/*. We 
have already seen on p. 319 that the true answer is (0.641¢/1)'/? and so we have come 
very close to it. 

Next we turn to the higher frequency. Here, a typical pattern of oscillation is not so 
obvious but, rather preempting the answer, we try 0. = —26,; we then obtain 2 = 9g/l 
and so conclude that the higher eigenfrequency > (9g/1)'/?. We have already seen that the 
exact answer is (9.359¢/I)'/* and so again we have come close to it. < 


A simplified version of the Rayleigh—Ritz method may be used to estimate the 
eigenvalues of a symmetric (or in general Hermitian) matrix B, the eigenvectors 
of which will be mutually orthogonal. By repeating the calculations leading to 
(9.18), A being replaced by the unit matrix |, it is easily verified that if 

x Bx 


AX) xx 


is evaluated for any vector x then 
At SAX) < Am, 


where 21, 4...,4m are the eigenvalues of B in order of increasing size. A similar 
result holds for Hermitian matrices. 


9.4 Exercises 


9.1 Three coupled pendulums swing perpendicularly to the horizontal line containing 
their points of suspension, and the following equations of motion are satisfied: 


—mxX; = cmx, + d(x; — x2), 
—MX2 = cMx2 + d(xz — X1) + d(x2 — x3), 
—mxX3 = cmx3 + d(x3 — X2), 


where x1, x2 and x3 are measured from the equilibrium points; m, M and m 
are the masses of the pendulum bobs; and c and d are positive constants. Find 
the normal frequencies of the system and sketch the corresponding patterns of 
oscillation. What happens as d > 0 or d > «? 

9.2 A double pendulum, smoothly pivoted at A, consists of two light rigid rods, AB 
and BC, each of length /, which are smoothly jointed at B and carry masses m and 
am at B and C respectively. The pendulum makes small oscillations in one plane 
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9:3 


9.4 


9.5, 


under gravity. At time t, AB and BC make angles 0@(t) and @(t), respectively, with 
the downward vertical. Find quadratic expressions for the kinetic and potential 
energies of the system and hence show that the normal modes have angular 
frequencies given by 

wo = . [i+o+ Var +o] : 
For « = 1/3, show that in one of the normal modes the mid-point of BC does 
not move during the motion. 
Continue the worked example, modelling a linear molecule, discussed at the end 
of section 9.1, for the case in which p = 2. 


(a) Show that the eigenvectors derived there have the expected orthogonality 
properties with respect to both A and B. 

(b) For the situation in which the atoms are released from rest with initial 
displacements x; = 2e, x. = —e and x; = 0, determine their subsequent 
motions and maximum displacements. 


Consider the circuit consisting of three equal capacitors and two different in- 
ductors shown in the figure. For charges Q; on the capacitors and currents I; 


a1 Q> 


| | 

| | 

Cc C 
Q; 

vor ==" L» 


I In 


“BOC 


through the components, write down Kirchhoff’s law for the total voltage change 
around each of two complete circuit loops. Note that, to within an unimportant 
constant, the conservation of current implies that Q; = Q; — Q. Express the loop 
equations in the form given in (9.7), namely 


AQ+BQ=0. 


Use this to show that the normal frequencies of the circuit are given by 


1 
o = CL [Li + Ly + (Li + L3 = Ly Ly)". 


Obtain the same matrices and result by finding the total energy stored in the 
various capacitors (typically Q?/(2C)) and in the inductors (typically LI?/2). 
For the special case L; = L2 = L determine the relevant eigenvectors and so 
describe the patterns of current flow in the circuit. 
It is shown in physics and engineering textbooks that circuits containing capaci- 
tors and inductors can be analysed by replacing a capacitor of capacitance C by a 
‘complex impedance’ 1/(i#C) and an inductor of inductance L by an impedance 
i@L, where @ is the angular frequency of the currents flowing and 7? = —1. 
Use this approach and Kirchhoff’s circuit laws to analyse the circuit shown in 
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9.6 


9.7 


the figure and obtain three linear equations governing the currents I), I> and 13. 
Show that the only possible frequencies of self-sustaining currents satisfy either 


I; 


P Q 


I, & 


(a) w?LC = 1 or (b) 3m?LC = 1. Find the corresponding current patterns and, 

in each case, by identifying parts of the circuit in which no current flows, draw 

an equivalent circuit that contains only one capacitor and one inductor. 

The simultaneous reduction to diagonal form of two real symmetric quadratic forms. 
Consider the two real symmetric quadratic forms u'Au and u'Bu, where u? 

stands for the row matrix (x y z), and denote by u” those column matrices 

that satisfy 


Bu” = 1,,Au", (E9.1) 
in which n is a label and the 2, are real, non-zero and all different. 


(a) By multiplying (E9.1) on the left by (u”)’, and the transpose of the corre- 
sponding equation for u” on the right by u”, show that (u”)'Au” = 0 for 
n#=m. 

(b) By noting that Au” = (/,)~'Bu", deduce that (u”)'Bu” = 0 for m # n. 

(c) It can be shown that the u” are linearly independent; the next step is to 
construct a matrix P whose columns are the vectors u”. 

(d) Make a change of variables u = Pv such that u' Au becomes v'Cv, and u’Bu 
becomes v'Dv. Show that C and D are diagonal by showing that cj = 0 if 
i# j, and similarly for djj. 


Thus u = Pv or v = P~!u reduces both quadratics to diagonal form. 
To summarise, the method is as follows: 


(a) find the 1, that allow (E9.1) a non-zero solution, by solving |B — AA| = 0; 
(b) for each 1, construct u"; 


(c) construct the non-singular matrix P whose columns are the vectors u”; 
(d) make the change of variable u = Pv. 


(It is recommended that the reader does not attempt this question until exercise 9.6 
has been studied.) 

If, in the pendulum system studied in section 9.1, the string is replaced by a 
second rod identical to the first then the expressions for the kinetic energy T and 
the potential energy V become (to second order in the 6;) 


T ~ MP ($0; +260,02 + 303) , 

V = Mgl (307 + 403). 
Determine the normal frequencies of the system and find new variables € and 7 
that will reduce these two expressions to diagonal form, i.e. to 


aye? + api? and by + bon’. 


331 


NORMAL MODES 


9.8 


9.9 


9.1 
9.3 


9.5, 


9.7 
9.9 


(It is recommended that the reader does not attempt this question until exercise 9.6 
has been studied.) 

Find a real linear transformation that simultaneously reduces the quadratic 
forms 


3x? + Sy? + 52? + 2yz + 62x — 2xy, 
5x? + 12y" + 8yz + 4zx 


to diagonal form. 

Three particles of mass m are attached to a light horizontal string having fixed 
ends, the string being thus divided into four equal portions each of length a and 
under a tension T. Show that for small transverse vibrations the amplitudes x’ 
of the normal modes satisfy Bx = (maw?/T)x, where B is the matrix 


2 -1 0 
-1 2 -1 
0 -1 2 


Estimate the lowest and highest eigenfrequencies using trial vectors (3 4 3)! 
T T 
and(3. —4 3)’. Use also the exact vectors (1 2 1) and (1 = afd 1) 
and compare the results. 
Use the Rayleigh—Ritz method to estimate the lowest oscillation frequency of a 
heavy chain of N links, each of length a (= L/N), which hangs freely from one 
end. (Try simple calculable configurations such as all links but one vertical, or 
all links collinear, etc.) 


9.5 Hints and answers 


See figure 9.6. 

(b) x, = e(cos wt + cos ./2ot), X2 = —€ COS /2ut, x3 = e(—cos at + cos »/2ut). 
At various times the three displacements will reach 2e, e, 2e respectively. For exam- 
ple, x; can be written as 2e cos[(./2— lot/2] cos[(./2+ 1)wt/2], i.e. an oscillation 
of angular frequency (./2+1)w/2 and modulated amplitude 2e cos[(./2—1)wt/2]; 
the amplitude will reach 2e after a time ~ 4n/[o(./2 — 1). 

As the circuit loops contain no voltage sources, the equations are homogeneous, 
and so for a non-trivial solution the determinant of coefficients must vanish. 

(a) I, = 0, I = —I;; no current in PQ; equivalent to two separate circuits of 
capacitance C and inductance L. 

(b) I, = —2I, = —213; no current in TU; capacitance 3C/2 and inductance 2L. 
@ = (2.634¢/1)!/? or (0.3661¢/1)'/?; 0; = € +n, 0 = 1.431E — 2.0979. 

Estimated, 10/17 < Mam?/T < 58/17; exact, 2—./2 < Maw?/T <2+./2. 
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Figure 9.6 The normal modes, as viewed from above, of the coupled pendu- 


lums in example 9.1. 
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Vector calculus 


In chapter 7 we discussed the algebra of vectors, and in chapter 8 we considered 
how to transform one vector into another using a linear operator. In this chapter 
and the next we discuss the calculus of vectors, ic. the differentiation and 
integration both of vectors describing particular bodies, such as the velocity of 
a particle, and of vector fields, in which a vector is defined as a function of the 
coordinates throughout some volume (one-, two- or three-dimensional). Since the 
aim of this chapter is to develop methods for handling multi-dimensional physical 
situations, we will assume throughout that the functions with which we have to 
deal have sufficiently amenable mathematical properties, in particular that they 
are continuous and differentiable. 


10.1 Differentiation of vectors 


Let us consider a vector a that is a function of a scalar variable u. By this 
we mean that with each value of u we associate a vector a(u). For example, in 
Cartesian coordinates a(u) = a,(u)i + a,(u)j + a_(u)k, where a,(u), ay(u) and a,(u) 
are scalar functions of u and are the components of the vector a(u) in the x-, y- 
and z- directions respectively. We note that if a(u) is continuous at some point 
u = uo then this implies that each of the Cartesian components a,(u), a,(u) and 
a,(u) is also continuous there. 

Let us consider the derivative of the vector function a(u) with respect to u. 
The derivative of a vector function is defined in a similar manner to the ordinary 
derivative of a scalar function f(x) given in chapter 2. The small change in 
the vector a(u) resulting from a small change Au in the value of u is given by 
Aa = a(u+ Au) — a(u) (see figure 10.1). The derivative of a(u) with respect to u is 
defined to be 


(10.1) 


10.1 DIFFERENTIATION OF VECTORS 


=a(u + Au) —a(u) 


Figure 10.1 A small change in a vector a(u) resulting from a small change 
in wu. 


assuming that the limit exists, in which case a(u) is said to be differentiable at 
that point. Note that da/du is also a vector, which is not, in general, parallel to 
a(u). In Cartesian coordinates, the derivative of the vector a(u) = ayi+ a,j + a-k 
is given by 


da da,, 
= i 


day. | da; 
du du I 


k. 
du du 
Perhaps the simplest application of the above is to finding the velocity and 
acceleration of a particle in classical mechanics. If the time-dependent position 
vector of the particle with respect to the origin in Cartesian coordinates is given 
by r(t) = x(t)hi+ y(j+2(0k then the velocity of the particle is given by the vector 
dr dx, dy. dz 


a ae oper es ae 


The direction of the velocity vector is along the tangent to the path r(t) at the 
instantaneous position of the particle, and its magnitude |v(t)| is equal to the 
speed of the particle. The acceleration of the particle is given in a similar manner 
by 

x, ay, @z 


dv 
= = t k. 
ate) dt ; dt? + dt? 


>The position vector of a particle at time t in Cartesian coordinates is given by r(t) = 
2i + (3t — 2)j + (30 — 1)k. Find the speed of the particle at t = 1 and the component of| 
its acceleration in the direction s =i+ 2j+k. 


The velocity and acceleration of the ee are given by 
v(t) = — = 4tit+ 3j+ 6tk, 


a(t) = = 4i + 6k. 
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x 


Figure 10.2 Unit basis vectors for two-dimensional Cartesian and plane polar 
coordinates. 


The speed of the particle at t = 1 is simply 


lw) = V2 +324+6 = 61. 


The acceleration of the particle is constant (ie. independent of t), and its component in 
the direction s is given by 


a (4i + 6k) - (i+ 2j+k) 56 4 


242241? 3 


Note that in the case discussed above i, j and k are fixed, time-independent 
basis vectors. This may not be true of basis vectors in general; when we are 
not using Cartesian coordinates the basis vectors themselves must also be dif- 
ferentiated. We discuss basis vectors for non-Cartesian coordinate systems in 
detail in section 10.10. Nevertheless, as a simple example, let us now consider 
two-dimensional plane polar coordinates p, ¢. 

Referring to figure 10.2, imagine holding ¢ fixed and moving radially outwards, 
i.e. in the direction of increasing p. Let us denote the unit vector in this direction 
by é,. Similarly, imagine keeping p fixed and moving around a circle of fixed radius 
in the direction of increasing ¢. Let us denote the unit vector tangent to the circle 
by @y. The two vectors €, and é, are the basis vectors for this two-dimensional 
coordinate system, just as i and j are basis vectors for two-dimensional Cartesian 
coordinates. All these basis vectors are shown in figure 10.2. 

An important difference between the two sets of basis vectors is that, while 
i and j are constant in magnitude and direction, the vectors é, and € have 
constant magnitudes but their directions change as p and @ vary. Therefore, 
when calculating the derivative of a vector written in polar coordinates we must 
also differentiate the basis vectors. One way of doing this is to express €, and 
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in terms of i and j. From figure 10.2, we see that 
é, = cos Pit sin pj, 
és = —sin dit cos dj. 


Since i and j are constant vectors, we find that the derivatives of the basis vectors 
é, and €4 with respect to t are given by 


de. dd, dp. 4. 

Tt sind an cos @ a peg, (10.2) 
dey dp. . dd, 4, 

a cos 7 sind Ht! pe, (10.3) 


where the overdot is the conventional notation for differentiation with respect to 
time. 


> The position vector of a particle in plane polar coordinates is x(t) = p(t)€,. Find expres- 
sions for the velocity and acceleration of the particle in these coordinates. 


Using result (10.4) below, the velocity of the particle is given by 
v(t) = #(t) = pb, + pb, = pe, + phey, 
where we have used (10.2). In a similar way its acceleration is given by 
ds ha 
a(t) = 7 (pe, + 0b €5) 

= pe + pep + phey + pbey + phey 

= pb, + (Hey) + pb(—46,) + pbes + phéy 

= (p— pb’)@, + (pb + 2p) ey. < 


Here we have used (10.2) and (10.3). 


10.1.1 Differentiation of composite vector expressions 


In composite vector expressions each of the vectors or scalars involved may be 
a function of some scalar variable u, as we have seen. The derivatives of such 
expressions are easily found using the definition (10.1) and the rules of ordinary 
differential calculus. They may be summarised by the following, in which we 
assume that a and b are differentiable vector functions of a scalar u and that ¢ 
is a differentiable scalar function of u: 


d db da 
ne )=a- a +7, > (10.5) 
AS eg (10.6) 
du du 
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The order of the factors in the terms on the RHS of (10.6) is, of course, just as 
important as it is in the original vector product. 


>A particle of mass m with position vector r relative to some origin O experiences a force 
F, which produces a torque (moment) T =r x F about O. The angular momentum of the 
particle about O is given by L =r x mv, where v is the particle’s velocity. Show that the 
rate of change of angular momentum is equal to the applied torque. 


The rate of change of angular momentum is given by 
dh d 
rie ae x my). 

Using (10.6) we obtain 


dy bt pt nm to 
dt = dt mv r it mv 


=vxmv + rx ee 
dt 
=04+ rxF=T, 
where in the last line we use Newton’s second law, namely F = d(mv)/dt. < 
If a vector a is a function of a scalar variable s that is itself a function of u, so 
that s = s(u), then the chain rule (see subsection 2.1.3) gives 
da(s)  dsda 
du duds’ 
The derivatives of more complicated vector expressions may be found by repeated 


application of the above equations. 
One further useful result can be derived by considering the derivative 


(10.7) 


da 
_ =2a-— 
1 ey) : du’ 
since a- a = a’, where a = |al, we see that 
d: vase Js 
a- =0 if a is constant. (10.8) 
u 


In other words, if a vector a(u) has a constant magnitude as u varies then it is 
perpendicular to the vector da/du. 


10.1.2 Differential of a vector 


As a final note on the differentiation of vectors, we can also define the differential 
of a vector, in a similar way to that of a scalar in ordinary differential calculus. 
In the definition of the vector derivative (10.1), we used the notion of a small 
change Aa in a vector a(u) resulting from a small change Au in its argument. In 
the limit Au > 0, the change in a becomes infinitesimally small, and we denote it 
by the differential da. From (10.1) we see that the differential is given by 


da = — du. (10.9) 


10.2 INTEGRATION OF VECTORS 


Note that the differential of a vector is also a vector. As an example, the 
infinitesimal change in the position vector of a particle in an infinitesimal time dt is 


dr = a at = vdt, 


where v is the particle’s velocity. 


10.2 Integration of vectors 


The integration of a vector (or of an expression involving vectors that may itself 
be either a vector or scalar) with respect to a scalar u can be regarded as the 
inverse of differentiation. We must remember, however, that 


(i) the integral has the same nature (vector or scalar) as the integrand, 
(ii) the constant of integration for indefinite integrals must be of the same 
nature as the integral. 


For example, if a(u) = d[A(u)]/du then the indefinite integral of a(u) is given by 


[aw du = A(u) +b, 


where b is a constant vector. The definite integral of a(u) from u = uy; to u = up 
is given by 


i: a(u) du = A(uz) — A(u)). 


>A small particle of mass m orbits a much larger mass M centred at the origin O. According 
to Newton's law of gravitation, the position vector r of the small mass obeys the differential 
equation 


mat a GMn . 
d2@ 


Show that the vector r x dr/dt is a constant of the motion. 


Forming the vector product of the differential equation with r, we obtain 


3 ar GM . 
rx — =-—rx fk. 
dt2 r2 


Since r and f are collinear, r x f = 0 and therefore we have 


rx Fee (10.10) 
However, 
d dr fr dr dr 
dt («x 5) Xe a a 0, 
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Figure 10.3. The unit tangent t, normal fi and binormal b to the space curve C 
at a particular point P. 


since the first term is zero by (10.10), and the second is zero because it is the vector product 
of two parallel (in this case identical) vectors. Integrating, we obtain the required result 


rx —=¢, (10.11) 


where ¢ is a constant vector. 

As a further point of interest we may note that in an infinitesimal time dt the change 
in the position vector of the small mass is dr and the element of area swept out by the 
position vector of the particle is simply dA = 3\r x dr|. Dividing both sides of this equation 
by dt, we conclude that 


dA 1 e dr |e| 
ge 2 ae 32 
and that the physical interpretation of the above result (10.11) is that the position vector r 


of the small mass sweeps out equal areas in equal times. This result is in fact valid for 
motion under any force that acts along the line joining the two particles. < 


10.3 Space curves 


In the previous section we mentioned that the velocity vector of a particle is a 
tangent to the curve in space along which the particle moves. We now give a more 
complete discussion of curves in space and also a discussion of the geometrical 
interpretation of the vector derivative. 

A curve C in space can be described by the vector r(u) joining the origin O of 
a coordinate system to a point on the curve (see figure 10.3). As the parameter u 
varies, the end-point of the vector moves along the curve. In Cartesian coordinates, 


r(u) = x(u)i+ y(u)j + z(wk, 


where x = x(u), y = y(u) and z = z(u) are the parametric equations of the curve. 
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This parametric representation can be very useful, particularly in mechanics when 
the parameter may be the time ft. We can, however, also represent a space curve 
by y = f(x), z = g(x), which can be easily converted into the above parametric 
form by setting u = x, so that 


r(u) = ui + f(wj + gk. 


Alternatively, a space curve can be represented in the form F(x,y,z) = 0, 
G(x,y,z) = 0, where each equation represents a surface and the curve is the 
intersection of the two surfaces. 

A curve may sometimes be described in parametric form by the vector r(s), 
where the parameter s is the arc length along the curve measured from a fixed 
point. Even when the curve is expressed in terms of some other parameter, it is 
straightforward to find the arc length between any two points on the curve. For 
the curve described by r(u), let us consider an infinitesimal vector displacement 


dr = dxi+dyj+dzk 
along the curve. The square of the infinitesimal distance moved is then given by 
(ds)? = dr - dr = (dx)? + (dy) + (dz), 

from which it can be shown that 

ds\? _ dr dr 

du) ~ du du 
Therefore, the arc length between two points on the curve r(u), given by u = u, 
and u = uw, is 


uz 
s= f a ie (10.12) 


du du 


> A curve lying in the xy-plane is given by y = y(x), z = 0. Using (10.12), show that the 


arc length along the curve between x =a and x = b is given by s= sh 1+ y? dx, where 
y’ =dy/dx. 


Let us first represent the curve in parametric form by setting u = x, so that 
r(u) = ui+ y(u)j. 


Differentiating with respect to u, we find 


dr =q4 dy, 
du du” 
from which we obtain 
dr dr dy 2 


VECTOR CALCULUS 


Therefore, remembering that u = x, from (10.12) the arc length between x = a and x = b 


is given by 
z) 
cue = 
< du -[ jie (2) 


This result was derived using more elementary methods in *) 2.< 


If a curve C is described by r(u) then, by considering figures 10.1 and 10.3, we 
see that, at any given point on the curve, dr/du is a vector tangent to C at that 
point, in the direction of increasing u. In the special case where the parameter u 
is the arc length s along the curve then dr/ds is a unit tangent vector to C and is 
denoted by t. 

The rate at which the unit tangent t changes with respect to s is given by 
dt/ds, and its magnitude is defined as the curvature « of the curve C at a given 
point, 
at 


ds? 


=|7|= 


We can also define the quantity = 1/x, which is called the radius of curvature. 

Since t is of constant (unit) magnitude, it follows from (10.8) that it is perpen- 
dicular to dt/ds. The unit vector in the direction perpendicular to t is denoted 
by fi and is called the principal normal at the point. We therefore have 


dt _ 


KA. (10.13) 
ds 


The unit vector b = t x fi, which is perpendicular to the plane containing f 
and fi, is called the binormal to C. The vectors t, fi and 6 form a right-handed 
rectangular cooordinate system (or triad) at any given point on C (see figure 10.3). 
As s changes so that the point of interest moves along C, the triad of vectors also 
changes. 

The rate at which b changes with respect to s is given by db/ds and is a 
measure of the torsion t of the curve at any given point. Since b is of constant 
magnitude, from (10.8) it is perpendicular to db/ds. We may further show that 
db/ds is also perpendicular to t, as follows. By definition b- t = 0, which on 
differentiating yields 


> 


Eps ig: di 


ds ds ds 
=F it 6 nA 
_ ab , 
“ds” 


where we have used the fact that b- fi = 0. Hence, since db/ds is perpendicular 
to both b and t, we must have db/ds oc fi. The constant of proportionality is —r, 
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so we finally obtain 
db =—th (10.14) 
ds 

Taking the dot product of each side with fi, we see that the torsion of a curve is 

given by A 

—_ « db 

T=—fh-—. 

We may also define the quantity 0 = 1/t, which is called the radius of torsion. 

Finally, we consider the derivative dii/ds. Since fi = b x t we have 


ds ds + Bx ds 
=—taxt + bx«kft 
=tbh—xt. (10.15) 


In summary, t, fi and b and their derivatives with respect to s are related to one 
another by the relations (10.13), (10.14) and (10.15), the Frenet—Serret formulae, 
dt di. » db . 

= n b —_ 


en dees —Kt een eee (10.16) 


where v is the speed of the particle, t is the unit tangent to the trajectory, fi is its principal 
normal and p is its radius of curvature. 


The velocity of the particle is given by 
Oe dr __drds_ ds» 


—————— t, 

dt dsdt at 
where ds/dt is the speed of the particle, which we denote by v, and t is the unit vector 
tangent to the trajectory. Writing the velocity as v = vt, and differentiating once more 
with respect to time t, we obtain 


a(t) = ay = doy, at. 
dt dt dt’ 
but we note that 
dt dsdt : 
i dds 
Therefore, we have 
dv. v, 
a(t) = a ri nl 


This shows that in addition to an acceleration dv/dt along the tangent to the particle’s 
trajectory, there is also an acceleration v*/p in the direction of the principal normal. The 
latter is often called the centripetal acceleration. < 
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Finally, we note that a curve r(u) representing the trajectory of a particle may 
sometimes be given in terms of some parameter u that is not necessarily equal to 
the time t but is functionally related to it in some way. In this case the velocity 
of the particle is given by 

_ dr drdu 
~ dt dudt’ 


Differentiating again with respect to time gives the acceleration as 
dvd (drdu\ — @r (du\* 4, edu 
ae dt dt \du dt du? \ dt du dt? 


10.4 Vector functions of several arguments 


The concept of the derivative of a vector is easily extended to cases where the 
vectors (or scalars) are functions of more than one independent scalar variable, 
U1, U2,...,Un. In this case, the results of subsection 10.1.1 are still valid, except 
that the derivatives become partial derivatives da/Cu; defined as in ordinary 
differential calculus. For example, in Cartesian coordinates, 


Oa day, Ody, az 
bu = duit Guit Gu 
In particular, (10.7) generalises to the chain rule of partial differentiation discussed 
in section 5.5. If a = a(uj,u2,...,u,) and each of the u; is also a function 
uj(V1,02,...,0n) of the variables v; then, generalising (5.17), 
Ga _ da du, da dus) | Ca Oty 2 da uj (10.17) 
Ov; Ou, Ov; ~ OUuy OD; Ou, Ov; ra Ou; Ov; 


A special case of this rule arises when a is an explicit function of some variable 
v, as well as of scalars uj,uo,...,u, that are themselves functions of v; then we 
have 
n 
da 0a da Ou; 


eae (10.18) 


dv é6v. — Gu; dv" 
j=l! 


We may also extend the concept of the differential of a vector given in (10.9) 


to vectors dependent on several variables 1, u2,...,Un: 
Oa Oa da ". da 
da = — du; + — dum +---+—du, = duj. (10.19) 
Out Our OUn a ou; 


As an example, the infinitesimal change in an electric field E in moving from a 
position r to a neighbouring one r+ dr is given by 


dE = 


0 oE oE 
i dx 4 By dy 4 oe dz. (10.20) 
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Figure 10.4 The tangent plane T to a surface S at a particular point P; 
u=c, and v = c, are the coordinate curves, shown by dotted lines, that pass 
through P. The broken line shows some particular parametric curve r = r(A) 
lying in the surface. 


10.5 Surfaces 


A surface S in space can be described by the vector r(u,v) joining the origin O of 
a coordinate system to a point on the surface (see figure 10.4). As the parameters 
u and v vary, the end-point of the vector moves over the surface. This is very 
similar to the parametric representation r(u) of a curve, discussed in section 10.3, 
but with the important difference that we require two parameters to describe a 
surface, whereas we need only one to describe a curve. 

In Cartesian coordinates the surface is given by 


r(u,v) = x(u,v)i + y(u, v)j + z(u, v)k, 


where x = x(u,v), y = y(u,v) and z = z(u,v) are the parametric equations of the 
surface. We can also represent a surface by z = f(x,y) or g(x, y,z) = 0. Either 
of these representations can be converted into the parametric form in a similar 
manner to that used for equations of curves. For example, if z = f(x,y) then by 
setting u = x and v = y the surface can be represented in parametric form by 


r(u,v) = uit vj + f(u,v)k. 


Any curve r(A), where 4 is a parameter, on the surface S can be represented 
by a pair of equations relating the parameters u and v, for example u = f(A) 
and v = g(A). A parametric representation of the curve can easily be found by 
straightforward substitution, ie. r(4) = r(u(4), v(A)). Using (10.17) for the case 
where the vector is a function of a single variable 2 so that the LHS becomes a 
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total derivative, the tangent to the curve r(A) at any point is given by 


dr or du or dv 
di dudda dvdid 


(10.21) 


The two curves u = constant and v = constant passing through any point P 
on S are called coordinate curves. For the curve u = constant, for example, we 
have du/di = 0, and so from (10.21) its tangent vector is in the direction dr/dv. 
Similarly, the tangent vector to the curve v = constant is in the direction dr/du. 

If the surface is smooth then at any point P on S the vectors dr/du and 
or/dv are linearly independent and define the tangent plane T at the point P (see 
figure 10.4). A vector normal to the surface at P is given by 


n=—xX—. (10.22) 


In the neighbourhood of P, an infinitesimal vector displacement dr is written 


or or 
dr = — du+ dv. 
ou Ov 
The element of area at P, an infinitesimal parallelogram whose sides are the 
coordinate curves, has magnitude 


A 7 4 F 
dS =| dux Sa) =|S x S) dudv = |n\ dude. (10.23) 
ou Ov Ou Ov 
Thus the total area of the surface is 
0 0 
A /I x SF) dudv = II in| du dv, (10.24) 
R|Ou Ov R 


where R is the region in the uv-plane corresponding to the range of parameter 
values that define the surface. 


> Find the element of area on the surface of a sphere of radius a, and hence calculate the 
total surface area of the sphere. 


We can represent a point r on the surface of the sphere in terms of the two parameters 0 
and ¢: 


r(0, d) = asin@cos i+ asin 0 sin dj + acos Ok, 


where @ and ¢ are the polar and azimuthal angles respectively. At any point P, vectors 
tangent to the coordinate curves 0 = constant and @ = constant are 


0 

5 =acos0cos ¢i+acos@sin¢j—asin 6k, 
or ; ae See : A 

a6 = —asin0sin di+asin@ cos dj. 
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A normal n to the surface at this point is then given by 


s i j k 
a acos@cos@ acosOsing —asind 
: / —asinOsing asinOcos¢ 0 


=a’ sin 0(sin # cos i+ sind sin @j + cos 0k), 
which has a magnitude of a” sin@. Therefore, the element of area at P is, from (10.23), 
dS = a’ sin0 dé dd, 


and the total surface area of the sphere is given by 


T 2n 
A= | ao | dé a’ sin = 4na’. 
0 0 


This familiar result can, of course, be proved by much simpler methods! < 


10.6 Scalar and vector fields 


We now turn to the case where a particular scalar or vector quantity is defined 
not just at a point in space but continuously as a field throughout some region 
of space R (which is often the whole space). Although the concept of a field is 
valid for spaces with an arbitrary number of dimensions, in the remainder of this 
chapter we will restrict our attention to the familiar three-dimensional case. A 
scalar field (x,y,z) associates a scalar with each point in R, while a vector field 
a(x, y,Z) associates a vector with each point. In what follows, we will assume that 
the variation in the scalar or vector field from point to point is both continuous 
and differentiable in R. 

Simple examples of scalar fields include the pressure at each point in a fluid 
and the electrostatic potential at each point in space in the presence of an electric 
charge. Vector fields relating to the same physical systems are the velocity vector 
in a fluid (giving the local speed and direction of the flow) and the electric field. 

With the study of continuously varying scalar and vector fields there arises the 
need to consider their derivatives and also the integration of field quantities along 
lines, over surfaces and throughout volumes in the field. We defer the discussion 
of line, surface and volume integrals until the next chapter, and in the remainder 
of this chapter we concentrate on the definition of vector differential operators 
and their properties. 


10.7 Vector operators 


Certain differential operations may be performed on scalar and vector fields 
and have wide-ranging applications in the physical sciences. The most important 
operations are those of finding the gradient of a scalar field and the divergence 
and curl of a vector field. It is usual to define these operators from a strictly 
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mathematical point of view, as we do below. In the following chapter, however, we 
will discuss their geometrical definitions, which rely on the concept of integrating 
vector quantities along lines and over surfaces. 
Central to all these differential operations is the vector operator V, which is 
called del (or sometimes nabla) and in Cartesian coordinates is defined by 
0 0 0 
V=i— +j—+k—. 10.25 
ag Nag. oz (ee) 
The form of this operator in non-Cartesian coordinate systems is discussed in 
sections 10.9 and 10.10. 


10.7.1 Gradient of a scalar field 


The gradient of a scalar field (x, y,z) is defined by 


Op | 0p 1 Op 
dg= = + k—. 10.26 
grad 6 = Vo lag tay Os (10.26) 
Clearly, V@ is a vector field whose x-, y- and z- components are the first partial 
derivatives of (x, y,z) with respect to x, y and z respectively. Also note that the 


vector field Vf should not be confused with the vector operator #V, which has 
components (¢ 0/0x, b 0/0y, b 0/02). 


> Find the gradient of the scalar field 6 = xy?z°. 


From (10.26) the gradient of @ is given by 
Vo = y?23i + 2xyz3j + 3xy?z’k. < 


The gradient of a scalar field @ has some interesting geometrical properties. 
Let us first consider the problem of calculating the rate of change of ¢ in some 
particular direction. For an infinitesimal vector displacement dr, forming its scalar 
product with V@ we obtain 


ob 4 
Vo-dr = (i fl Ke) (idx + jdy +kdx), 
Ox oy Oz 
8b, a ao 
rig Oe aye ag 
mr (10.27) 


which is the infinitesimal change in # in going from position r to r+dr. In 
particular, if r depends on some parameter u such that r(u) defines a space curve 
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Vo. 


Q 


0 
ap in the direction a 
ds 


o = constant 


Figure 10.5 Geometrical properties of Vd. PQ gives the value of dp/ds in 
the direction a. 


then the total derivative of @ with respect to u along the curve is simply 


dp dr 
— =Vo-—. 10.28 
du ue du ( ) 
In the particular case where the parameter u is the arc length s along the curve, 


the total derivative of @ with respect to s along the curve is given by 


do * 
—=V¢-t, 10.29 
= Ve (10.29) 
where t is the unit tangent to the curve at the given point, as discussed in 
section 10.3. 

In general, the rate of change of @ with respect to the distance s in a particular 
direction a is given by 


dd 
—_—= a 10.30 
Ae Vo: a ( ) 
and is called the directional derivative. Since 4 is a unit vector we have 


ae = |V¢| cos 0 

ds 

where @ is the angle between a and V@ as shown in figure 10.5. Clearly V@ lies 
in the direction of the fastest increase in ¢, and |V@| is the largest possible value 
of dp/ds. Similarly, the largest rate of decrease of ¢ is df/ds = —|Vd| in the 
direction of —V¢. 
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> For the function @ = x?y + yz at the point (1,2,—1), find its rate of change with distance 
in the direction a = i+ 2j + 3k. At this same point, what is the greatest possible rate of| 
change with distance and in which direction does it occur? 


The gradient of @ is given by (10.26): 
Vp = 2xyit (x? + 2)j + yk, 
=4i+2k at the point (1,2,—1). 
The unit vector in the direction of a is a= ali + 2j-+ 3k), so the rate of change of # 
with distance s in this direction is, using (10.30), 


dp 7 1 10 
7 Yea qatt9 Rare 


From the above discussion, at the point (1,2,—1) dd/ds will be greatest in the direction 
of Vd = 4i+ 2k and has the value |Vd| = 4/20 in this direction. < 


We can extend the above analysis to find the rate of change of a vector 
field (rather than a scalar field as above) in a particular direction. The scalar 
differential operator 4-V can be shown to give the rate of change with distance 
in the direction a4 of the quantity (vector or scalar) on which it acts. In Cartesian 


coordinates it may be written as 


a-V= 


a 
ayz— +a (10.31) 
Ox 
Thus we can write the infinitesimal change in an electric field in moving from r 

to r+ dr given in (10.20) as dE = (dr- V)E. 

A second interesting geometrical property of Vf may be found by considering 
the surface defined by (x,y,z) = c, where c is some constant. If t is a unit 
tangent to this surface at some point then clearly dé/ds = 0 in this direction 
and from (10.29) we have V@- t = 0. In other words, V@ is a vector normal to 
the surface (x,y,z) = ¢ at every point, as shown in figure 10.5. If fi is a unit 
normal to the surface in the direction of increasing (x, y,z), then the gradient is 
sometimes written 


Woe oR (10.32) 


where 0¢/0n = |Vd| is the rate of change of ¢ in the direction fi and is called 
the normal derivative. 


> Find expressions for the equations of the tangent plane and the line normal to the surface 
(x, y,z) = at the point P with coordinates xo, yo, 20. Use the results to find the equations 
of the tangent plane and the line normal to the surface of the sphere @ = x? +y*+z7 = a@ 
at the point (0,0, a). 


A vector normal to the surface (x, y,z) = c at the point P is simply V@ evaluated at that 
point; we denote it by no. If ro is the position vector of the point P relative to the origin, 
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gartyt2=a 
X 


Figure 10.6 The tangent plane and the normal to the surface of the sphere 
o=x?+y?4+2? =a at the point ro with coordinates (0,0, a). 


and r is the position vector of any point on the tangent plane, then the vector equation of 
the tangent plane is, from (7.41), 


(r —1o)- Mo = 0. 


Similarly, if r is the position vector of any point on the straight line passing through P 
(with position vector ro) in the direction of the normal no then the vector equation of this 
line is, from subsection 7.7.1, 


(r—ro) X no = 0. 
For the surface of the sphere # = x? ++ y? +27 =a’, 
Vo = 2xi+ 2yj + 2zk 
=2ak at the point (0,0, a). 
Therefore the equation of the tangent plane to the sphere at this point is 
(r —ro)- 2ak = 0. 


This gives 2a(z — a) = 0 or z = a, as expected. The equation of the line normal to the 
sphere at the point (0,0, a) is 


(r —ro) X 2ak = 0, 


which gives 2ayi — 2axj = 0 or x = y = 0, ie. the z-axis, as expected. The tangent plane 
and normal to the surface of the sphere at this point are shown in figure 10.6. < 


Further properties of the gradient operation, which are analogous to those of 
the ordinary derivative, are listed in subsection 10.8.1 and may be easily proved. 
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In addition to these, we note that the gradient operation also obeys the chain 
rule as in ordinary differential calculus, i.e. if @ and w are scalar fields in some 
region R then 


V[ow)] = sve. 


10.7.2 Divergence of a vector field 
The divergence of a vector field a(x, y,z) is defined by 


. 
Oa, . Cay | Oa, 


di = . = == —— 
iva=V-a ae) Gy ee? 


(10.33) 


where ax, ay and a, are the x-, y- and z- components of a. Clearly, V-a is a scalar 
field. Any vector field a for which V- a = 0 is said to be solenoidal. 


> Find the divergence of the vector field a = x*y?i+ y?z?j + x°z7k. 


From (10.33) the divergence of a is given by 
V-a=2xy? + 2yz? + 2x°z = Axy? + yz? +x°z). 


We will discuss fully the geometric definition of divergence and its physical 
meaning in the next chapter. For the moment, we merely note that the divergence 
can be considered as a quantitative measure of how much a vector field diverges 
(spreads out) or converges at any given point. For example, if we consider the 
vector field v(x, y,z) describing the local velocity at any point in a fluid then V-v 
is equal to the net rate of outflow of fluid per unit volume, evaluated at a point 
(by letting a small volume at that point tend to zero). 

Now if some vector field a is itself derived from a scalar field via a = Vd then 
V-a has the form V - V@ or, as it is usually written, V2, where V? (del squared) 
is the scalar differential operator 


(10.34) 


V@ is called the Laplacian of @ and appears in several important partial differ- 
ential equations of mathematical physics, discussed in chapters 20 and 21. 


> Find the Laplacian of the scalar field 6 = xy?z?. 


From (10.34) the Laplacian of ¢ is given by 


oe @? oe 
db HH 
Ox? Oy? 62? 


Vb 2xz3 + 6xy7z. 
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10.7.3 Curl of a vector field 


The curl of a vector field a(x, y,z) is defined by 


Oa, Oa, oa Oa Oa, Oa, 
1 —_ = Z y ° ‘Xe Z . y XG k 
Mees (F eit (F mit (F =) 


where ay, ay and a, are the x-, y- and z- components of a. The RHS can be 
written in a more memorable form as a determinant: 


oe ae 
0 0 0 


a ay a, 
where it is understood that, on expanding the determinant, the partial derivatives 
in the second row act on the components of a in the third row. Clearly, V x a is 
itself a vector field. Any vector field a for which V x a = 0 is said to be irrotational. 


> Find the curl of the vector field a = x*y?z?i+ y?z*j + x?27k. 


The curl of a is given by 


i j k 
é 6 6 2 DLO ON eos aD 
Vo=| a Sy (be = —2 [yzit (xz? — x? y?z)j + x7 yz"k]. < 
xyz? y2z2 x2p2 


For a vector field v(x, y,z) describing the local velocity at any point in a fluid, 
V x v is a measure of the angular velocity of the fluid in the neighbourhood of 
that point. If a small paddle wheel were placed at various points in the fluid then 
it would tend to rotate in regions where V x v ¥ 0, while it would not rotate in 
regions where V x v= 0. 

Another insight into the physical interpretation of the curl operator is gained 
by considering the vector field v describing the velocity at any point in a rigid 
body rotating about some axis with angular velocity w. If r is the position vector 
of the point with respect to some origin on the axis of rotation then the velocity 
of the point is given by v= @ xr. Without any loss of generality, we may take 
w to lie along the z-axis of our coordinate system, so that @ = wk. The velocity 
field is then v= —wyi+ xj. The curl of this vector field is easily found to be 


i j 
a 060 6 
Vxv ay ay ae 2ok = 2. (10.36) 
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Vio+yp) = ae 
V-(at+b)=V-a+V-b 
V x (a+b) = ene an 
V(op) = Vp + Vo 
V(a-b) =ax(Vxb)+bx(Vxa)+(a-V)b+(b-V)a 
V- (ga) = dV-at+a-Voh 


V-(axb) =b-(V x a)—a-(V xb) 
V x (pa) =VhPxat+oVxa 
V x (ax b) = a(V-b)—b(V -a) + (b- V)a—(a- V)b 


Table 10.1 Vector operators acting on sums and products. The operator V is 
defined in (10.25); @ and yp are scalar fields, a and b are vector fields. 


Therefore the curl of the velocity field is a vector equal to twice the angular 
velocity vector of the rigid body about its axis of rotation. We give a full 
geometrical discussion of the curl of a vector in the next chapter. 


10.8 Vector operator formulae 


In the same way as for ordinary vectors (chapter 7), for vector operators certain 
identities exist. In addition, we must consider various relations involving the 
action of vector operators on sums and products of scalar and vector fields. Some 
of these relations have been mentioned earlier, but we list all the most important 
ones here for convenience. The validity of these relations may be easily verified 
by direct calculation (a quick method of deriving them using tensor notation is 
given in chapter 26). 

Although some of the following vector relations are expressed in Cartesian 
coordinates, it may be proved that they are all independent of the choice of 
coordinate system. This is to be expected since grad, div and curl all have clear 
geometrical definitions, which are discussed more fully in the next chapter and 
which do not rely on any particular choice of coordinate system. 


10.8.1 Vector operators acting on sums and products 


Let ¢ and yp be scalar fields and a and b be vector fields. Assuming these fields 
are differentiable, the action of grad, div and curl on various sums and products 
of them is presented in table 10.1. 

These relations can be proved by direct calculation. 
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> Show that 
V x (ga) = Vo x a+ PV Xa. 


The x-component of the LHS is 


0 0 da, 0 day 0 
£ (a2) — (bay) = 6 + La 62 - Pay, 


_ (ba, ay ao ao 
e (= at) + (Ga Ba, ; 
= f(V Xa). + (Ve x a)x, 


where, for example, (Vd x a), denotes the x-component of the vector V@ x a. Incorporating 
the y- and z- components, which can be similarly found, we obtain the stated result. < 


Some useful special cases of the relations in table 10.1 are worth noting. If r is 
the position vector relative to some origin and r = |r|, then 


d 
Vo(r) = “ i, 
V-[H0r}] = 34(7) +r), 
2 = a p(r) 2 dd(r) 
Vi ¢(r) = qt eat 


V x [d(r)r] = 0. 


These results may be proved straightforwardly using Cartesian coordinates but 
far more simply using spherical polar coordinates, which are discussed in subsec- 
tion 10.9.2. Particular cases of these results are 


Vr=f, V-r=3, Vxr=0, 


together with 


V- (=) =-vV (+) = 4n6(r), 
if: r 


where 6(r) is the Dirac delta function, discussed in chapter 13. The last equation is 
important in the solution of certain partial differential equations and is discussed 
further in chapter 20. 


10.8.2. Combinations of grad, div and curl 


We now consider the action of two vector operators in succession on a scalar or 
vector field. We can immediately discard four of the nine obvious combinations of 
grad, div and curl, since they clearly do not make sense. If ¢ is a scalar field and 
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a is a vector field, these four combinations are grad(grad @), div(div a), curl(div a) 
and grad(curl a). In each case the second (outer) vector operator is acting on the 
wrong type of field, i.e. scalar instead of vector or vice versa. In grad(grad @), for 
example, grad acts on grad ¢, which is a vector field, but we know that grad only 
acts on scalar fields (although in fact we will see in chapter 26 that we can form 
the outer product of the del operator with a vector to give a tensor, but that need 
not concern us here). 

Of the five valid combinations of grad, div and curl, two are identically zero, 
namely 


curl grad 6 =V x Vo = 0, (10.37) 
div curla=V-(V x a)=0. (10.38) 


From (10.37), we see that if a is derived from the gradient of some scalar function 
such that a = V@ then it is necessarily irrotational (V x a = 0). We also note 
that if a is an irrotational vector field then another irrotational vector field is 
a+Vo-+c, where ¢ is any scalar field and ¢ is a constant vector. This follows 
since 


Vx(at+Vo+e)=Vxa+VxVG=0. 


Similarly, from (10.38) we may infer that if b is the curl of some vector field a 
such that b = V x a then b is solenoidal (V - b = 0). Obviously, if b is solenoidal 
and ¢ is any constant vector then b + ¢ is also solenoidal. 

The three remaining combinations of grad, div and curl are 


2p PH Oo 


+ 
0x2 Gy? az?” 


div grad 6 =V-VO=V'6 = 
grad diva = V(V - a), 


(10.39) 


a 07 ay 07a, ps 02a, \, 02a, 07a, Ca, \, 
dx? ' Oxdy | dxdz dydx ' dy? | dydz)3 
a Cay a, 
— -\k 10.40 
+ (= e dzdy re 022 ) : ( ) 
curl curla = V x (V x a) = V(V-a) —V’a, (10.41) 


where (10.39) and (10.40) are expressed in Cartesian coordinates. In (10.41), the 
term V7a has the linear differential operator V* acting on a vector (as opposed to 
a scalar as in (10.39)), which of course consists of a sum of unit vectors multiplied 
by components. Two cases arise. 


(i) If the unit vectors are constants (i.e. they are independent of the values of 
the coordinates) then the differential operator gives a non-zero contribution 
only when acting upon the components, the unit vectors being merely 
multipliers. 
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(ii) If the unit vectors vary as the values of the coordinates change (i.e. are 
not constant in direction throughout the whole space) then the derivatives 
of these vectors appear as contributions to V7a. 


Cartesian coordinates are an example of the first case in which each component 
satisfies (V7a); = V7a;. In this case (10.41) can be applied to each component 
separately: 


[V x (V x a)]; = [V(V-a)], — V7q;. (10.42) 


However, cylindrical and spherical polar coordinates come in the second class. 
For them (10.41) is still true, but the further step to (10.42) cannot be made. 

More complicated vector operator relations may be proved using the relations 
given above. 


> Show that 
V-(Vd x Vp) = 0, 


where ¢ and w are scalar fields. 


From the previous section we have 
V-(ax b) =b-(V x a)—a-(V Xb). 
If we let a = V@ and b = Vy then we obtain 
V-(V¢ x Vp) = Vp -(V x Vo) —Vo-(V x Vy) = 9, (10.43) 
since V x Vf =0=V x Vy, from (10.37). < 


10.9 Cylindrical and spherical polar coordinates 


The operators we have discussed in this chapter, ie. grad, div, curl and V?, 
have all been defined in terms of Cartesian coordinates, but for many physical 
situations other coordinate systems are more natural. For example, many systems, 
such as an isolated charge in space, have spherical symmetry and spherical polar 
coordinates would be the obvious choice. For axisymmetric systems, such as fluid 
flow in a pipe, cylindrical polar coordinates are the natural choice. The physical 
laws governing the behaviour of the systems are often expressed in terms of 
the vector operators we have been discussing, and so it is necessary to be able 
to express these operators in these other, non-Cartesian, coordinates. We first 
consider the two most common non-Cartesian coordinate systems, i.e. cylindrical 
and spherical polars, and go on to discuss general curvilinear coordinates in the 
next section. 


10.9.1 Cylindrical polar coordinates 


As shown in figure 10.7, the position of a point in space P having Cartesian 
coordinates x,y,z may be expressed in terms of cylindrical polar coordinates 
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p,,Z, where 
x=pcosd, y=psing, 2z=z, (10.44) 


and p > 0,0 < ¢@ < 2a and —w < z < ~. The position vector of P may therefore 
be written 


r=pcos¢it+psingj+zk. (10.45) 


If we take the partial derivatives of r with respect to p, ¢ and z respectively then 
we obtain the three vectors 


or : : 

e&, = ap =cos di+sin dj, (10.46) 
7 

ey = a =-—psin¢dit+ pcos dj, (10.47) 
A 

j= 2h. (10.48) 
Oz 


These vectors lie in the directions of increasing p, @ and z respectively but are 
not all of unit length. Although e,, eg and e, form a useful set of basis vectors 
in their own right (we will see in section 10.10 that such a basis is sometimes the 
most useful), it is usual to work with the corresponding unit vectors, which are 
obtained by dividing each vector by its modulus to give 


é, =e, = cos di+sin dj, (10.49) 
1 

és = —ey = —singdi+cos dj, (10.50) 
p 

je =k (10.51) 


These three unit vectors, like the Cartesian unit vectors i, j and k, form an 
orthonormal triad at each point in space, ie. the basis vectors are mutually 
orthogonal and of unit length (see figure 10.7). Unlike the fixed vectors i, j and k, 
however, €, and €y change direction as P moves. 

The expression for a general infinitesimal vector displacement dr in the position 
of P is given, from (10.19), by 


ey pitt 
ap? a6 
= dpe, +dbey +dze- 

= dpb, + pdbey + dzé.. (10.52) 


This expression illustrates an important difference between Cartesian and cylin- 
drical polar coordinates (or non-Cartesian coordinates in general). In Cartesian 
coordinates, the distance moved in going from x to x + dx, with y and z held 
constant, is simply ds = dx. However, in cylindrical polars, if @ changes by dd, 
with p and z held constant, then the distance moved is not dd, but ds = pdd. 
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oD 


Figure 10.7. Cylindrical polar coordinates p, ¢, z. 


Zz, 
pdp 
\ 1 


x 


Figure 10.8 The element of volume in cylindrical polar coordinates is given 
by pdp d¢ dz. 


Factors, such as the p in pd@, that multiply the coordinate differentials to give 
distances are known as scale factors. From (10.52), the scale factors for the p-, d- 
and z- coordinates are therefore 1, p and 1 respectively. 

The magnitude ds of the displacement dr is given in cylindrical polar coordinates 
by 


(ds)? = dr- dr = (dp)? + p°(dby? + (dz), 


where in the second equality we have used the fact that the basis vectors are 
orthonormal. We can also find the volume element in a cylindrical polar system 
(see figure 10.8) by calculating the volume of the infinitesimal parallelepiped 
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oO 
db = ake Oe eat 
V ae! F ape? ay 
e = 7 aie léag | 0a, 
eo Rept ob | Oz 
& pes @. 
1] 0 0 0 
vee p| op 06 oO 
dp pag a; 
Vo = ae aoe 1 ao ao 
pop \ 6p p? 0g? dz? 


Table 10.2 Vector operators in cylindrical polar coordinates; ® is a scalar 
field and a is a vector field. 


defined by the vectors dpé,, pdpéy and dzé-,: 
dV =|dpé, -(pdpéeg x dzé.)| = pdpdddz, 


which again uses the fact that the basis vectors are orthonormal. For a simple 
coordinate system such as cylindrical polars the expressions for (ds)? and dV are 
obvious from the geometry. 

We will now express the vector operators discussed in this chapter in terms 
of cylindrical polar coordinates. Let us consider a vector field a(p,¢,z) and a 
scalar field O(p, ¢,z), where we use ® for the scalar field to avoid confusion with 
the azimuthal angle @. We must first write the vector field in terms of the basis 
vectors of the cylindrical polar coordinate system, i.e. 


a = Ap €y + dy Cg + ze, 


where a,, ag and a, are the components of a in the p-, ¢- and z- directions 
respectively. The expressions for grad, div, curl and V? can then be calculated 
and are given in table 10.2. Since the derivations of these expressions are rather 
complicated we leave them until our discussion of general curvilinear coordinates 
in the next section; the reader could well postpone examination of these formal 
proofs until some experience of using the expressions has been gained. 


> Express the vector field a = yzi— yj+xz?k in cylindrical polar coordinates, and hence 


calculate its divergence. Show that the same result is obtained by evaluating the divergence 
in Cartesian coordinates. 


The basis vectors of the cylindrical polar coordinate system are given in (10.49)-(10.51). 
Solving these equations simultaneously for i, j and k we obtain 


i=cos $€, — sind ey 
j=sin dé, + cos p éy 
k =6.. 
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Figure 10.9 Spherical polar coordinates r, 0, ¢. 


Substituting these relations and (10.44) into the expression for a we find 
a = zpsin d (cos b&, — sing é,) — psin (sin dé, + cos bey) + 2’p cos He, 
= (zpsin pcos ¢ — psin’ #) é, — (zp sin’ + psin¢cos¢) é + 2?pcos Pé.. 
Substituting into the expression for V - a given in table 10.2, 
V-a=2zsin bcos # — 2sin’  — 2z sing cos ¢ — cos’ @ + sin’ + 2zpcos 
=2zpcos¢—1. 


Alternatively, and much more quickly in this case, we can calculate the divergence 
directly in Cartesian coordinates. We obtain 


A A 

Oa da Oa 

— + = = 2ex-1, 
Ox oy Oz 


V-a= 


which on substituting x = pcos@ yields the same result as the calculation in cylindrical 
polars. < 


Finally, we note that similar results can be obtained for (two-dimensional) 
polar coordinates in a plane by omitting the z-dependence. For example, (ds)* = 
(dp)? + p?(dd)*, while the element of volume is replaced by the element of area 
dA = pdp dd. 


10.9.2 Spherical polar coordinates 


As shown in figure 10.9, the position of a point in space P, with Cartesian 
coordinates x,y,z, may be expressed in terms of spherical polar coordinates 
r,0,@, where 


x=rsinOcos¢?, y=rsin0sing, z=rcosd, (10.53) 
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andr >0,0<0<7 and 0< ¢ < 2z. The position vector of P may therefore be 
written as 


r=rsin@cosdit+rsin@sindj+rcosdk. 


If, in a similar manner to that used in the previous section for cylindrical polars, 
we find the partial derivatives of r with respect to r, 0 and ¢ respectively and 
divide each of the resulting vectors by its modulus then we obtain the unit basis 
vectors 


é, = sin cos di+ sin @ sin dj + cos 0k, 
€9 = cos 0 cos i+ cos 0 sin dj — sink, 


é, = —sindi+cos¢j. 


These unit vectors are in the directions of increasing r, 0 and ¢ respectively 
and are the orthonormal basis set for spherical polar coordinates, as shown in 
figure 10.9. 

A general infinitesimal vector displacement in spherical polars is, from (10.19), 


dr = dré, +rd0@ +rsin0 dd ég; (10.54) 


thus the scale factors for the r-, 0- and @- coordinates are 1, r and rsin@ 
respectively. The magnitude ds of the displacement dr is given by 


(ds)? = dr- dr = (dry + (dO) +r? sin? 0(doy’, 


since the basis vectors form an orthonormal set. The element of volume in 
spherical polar coordinates (see figure 10.10) is the volume of the infinitesimal 
parallelepiped defined by the vectors dr é,, rd0 €g and r sin 0 dd € and is given by 


dV =|dré@,-(rd0@ x rsinOddég)| = r’ sin 0 dr dO dd, 


where again we use the fact that the basis vectors are orthonormal. The expres- 
sions for (ds) and dV in spherical polars can be obtained from the geometry of 
this coordinate system. 

We will now express the standard vector operators in spherical polar coordi- 
nates, using the same techniques as for cylindrical polar coordinates. We consider 
a scalar field O(r,0,) and a vector field a(r,0,). The latter may be written in 
terms of the basis vectors of the spherical polar coordinate system as 


a =a, é, + a9 @o + ag eg, 


where a,, ag and ag are the components of a in the r-, 6- and ¢- directions 
respectively. The expressions for grad, div, curl and V? are given in table 10.3. 
The derivations of these results are given in the next section. 

As a final note, we mention that, in the expression for V?® given in table 10.3, 
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Table 10.3 Vector operators in spherical polar coordinates; ® is a scalar field 
and a is a vector field. 


x 
Figure 10.10 The element of volume in spherical polar coordinates is given 


by r? sin @ dr dO dd. 


we can rewrite the first term on the RHS as follows: 


10 aii) 1 é? 
Ge a - (ro), 


r? or or 
which can often be useful in shortening calculations. 
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10.10 General curvilinear coordinates 


As indicated earlier, the contents of this section are more formal and technically 
complicated than hitherto. The section could be omitted until the reader has had 
some experience of using its results. 

Cylindrical and spherical polars are just two examples of what are called 
general curvilinear coordinates. In the general case, the position of a point P 
having Cartesian coordinates x,y,z may be expressed in terms of the three 
curvilinear coordinates u1, u2,u3, where 


X=X(uU1,U2,U3), y=V(U1,U2,uU3), 2 = 2(U1, U2, U3), 
and similarly 
uy = U(X, y,Z), U2 = U(X, y,Z), U3 = U3(X, y,Z). 


We assume that all these functions are continuous, differentiable and have a 
single-valued inverse, except perhaps at or on certain isolated points or lines, 
so that there is a one-to-one correspondence between the x, y, z and m4, wu, 13 
systems. The w-, u2- and u3- coordinate curves of a general curvilinear system 
are analogous to the x-, y- and z- axes of Cartesian coordinates. The surfaces 
Uy = Cy, Uy = Cy and uz = c3, where cy, C2, c3 are constants, are called the 
coordinate surfaces and each pair of these surfaces has its intersection in a curve 
called a coordinate curve or line (see figure 10.11). 

If at each point in space the three coordinate surfaces passing through the point 
meet at right angles then the curvilinear coordinate system is called orthogonal. 
For example, in spherical polars uj =r, uz = 0, u3 = ¢ and the three coordinate 
surfaces passing through the point (R,O,®) are the sphere r = R, the circular 
cone 0 = © and the plane ¢ = ®, which intersect at right angles at that 
point. Therefore spherical polars form an orthogonal coordinate system (as do 
cylindrical polars) . 

If r(ui, u2,u3) is the position vector of the point P then e; = dr/Ou; is a vector 
tangent to the u;-curve at P (for which u and u3 are constants) in the direction 
of increasing u;. Similarly, ey = ér/duz and e3 = Or/0u3 are vectors tangent to 
the u2- and u3- curves at P in the direction of increasing uy and u3 respectively. 
Denoting the lengths of these vectors by h,, hy and hz, the unit vectors in each of 
these directions are given by 

" lor ., lor ., 1 or 


eq = - eo =-—-— e=— 
hy Ou, hy Our’ h3 Ou3’ 


where h; = |ér/0us|, hy = |Or/Ouz| and h3 = |ér/du3|. 

The quantities h,, ho, hz are the scale factors of the curvilinear coordinate 
system. The element of distance associated with an infinitesimal change du; in 
one of the coordinates is h;du;. In the previous section we found that the scale 
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Figure 10.11 General curvilinear coordinates. 


factors for cylindrical and spherical polar coordinates were 


for cylindrical polars h,=1, hg=p, h, =1, 


for spherical polars h.=1, ho=r, hg =rsin0. 


Although the vectors e1, e2, e3 form a perfectly good basis for the curvilinear 
coordinate system, it is usual to work with the corresponding unit vectors €1, €2, 
é3. For an orthogonal curvilinear coordinate system these unit vectors form an 
orthonormal basis. 

An infinitesimal vector displacement in general curvilinear coordinates is given 
by, from (10.19), 


A 0 
yee MPA I LY 9 (10.55) 
ouy dun uz 
=> du, ee + dun @ + du; e3 (10.56) 
=> hy du, é + hy duy & = hy du; 63. (10.57) 


In the case of orthogonal curvilinear coordinates, where the é; are mutually 
perpendicular, the element of arc length is given by 


(ds)? = dr - dv = h} (du)? + h3(duz)? + 13 (du3)’. (10.58) 


The volume element for the coordinate system is the volume of the infinitesimal 
parallelepiped defined by the vectors (ér/0u;) du; = du; e; = h; du; €;, for i = 1, 2, 3. 


365 


VECTOR CALCULUS 


For orthogonal coordinates this is given by 


dV= |duy ei: (duz @2 xX du3 e3)| 
= |hy e : (hz © x h3 é3)| du, dur du3 


= hyhyh; du, dup du3. 


Now, in addition to the set {@;}, i = 1,2,3, there exists another useful set of 
three unit basis vectors at P. Since Vu; is a vector normal to the surface u, = cy, 
a unit vector in this direction is €; = Vuj/|Vui|. Similarly, €é: = Vuo/|Vu2| and 
é; = Vu3/|Vu3| are unit vectors normal to the surfaces uw. = c) and u3 = © 
respectively. 

Therefore at each point P in a curvilinear coordinate system, there exist, in 
general, two sets of unit vectors: {@;}, tangent to the coordinate curves, and {é;}, 
normal to the coordinate surfaces. A vector a can be written in terms of either 
set of unit vectors: 


a= aye; + do@ + 0363 = Aye, + Ar€o + A363, 


where aj, dz, a3 and A;, Ax, A3 are the components of a in the two systems. It 
may be shown that the two bases become identical if the coordinate system is 
orthogonal. 

Instead of the unit vectors discussed above, we could instead work directly with 
the two sets of vectors {e; = dr/Ou;} and {€; = Vu;}, which are not, in general, of 
unit length. We can then write a vector a as 


a = He; + He) + 0303 = Bie; + PrE2 + B33, 


or more explicitly as 


F 

or or or 

a= ou5— +%5— +035— = fiVu + BrVu2 + B3Vus, 
ou, ou Ou3 


where 0, #2, 73 and fj, fo, 63 are called the contravariant and covariant com- 
ponents of a respectively. A more detailed discussion of these components, in 
the context of tensor analysis, is given in chapter 26. The (in general) non-unit 
bases {ej} and {e;} are often the most natural bases in which to express vector 
quantities. 


[> Show that {e;} and {e;} are reciprocal systems of vectors. | 


Let us consider the scalar product e; - €;; using the Cartesian expressions for r and V, we 
obtain 


ej ep= — Vuj 
Ox oy 0z Ou; Ou; Ou; 
i : k). js js ik 
(Fit Fire ) (eis Bis ) 
Ox Ou; | Oy Ou; . Oz Ouj _ Ou; 


Ou; Ox Ou; Oy ~— Ou; Oz Ou; 
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In the last step we have used the chain rule for partial differentiation. Therefore e; -¢; = 1 
if i= j, and e;- e; = 0 otherwise. Hence {e;} and {e;} are reciprocal systems of vectors. < 


We now derive expressions for the standard vector operators in orthogonal 
curvilinear coordinates. Despite the useful properties of the non-unit bases dis- 
cussed above, the remainder of our discussion in this section will be in terms of 
the unit basis vectors {é;}. The expressions for the vector operators in cylindrical 
and spherical polar coordinates given in tables 10.2 and 10.3 respectively can be 
found from those derived below by inserting the appropriate scale factors. 


Gradient 


The change d® in a scalar field ® resulting from changes duj,du2,du3 in the 
coordinates u1, U2, u3 is given - from (5.5), 


0® 0® 
d® = = ee va. aoe + Pils du3. 


For orthogonal curvilinear coordinates u1,u2,u3 we find from (10.57), and com- 
parison with (10.27), that we can write this as 


d® =V- dr, (10.59) 


where V®@ is given by 


1 0® 1 0® 1 0@ 
yo- 19, 100, Lo, 10.60 
het WOU ua 0) 


This implies that the del operator can be written 


é 0 é 0 6; 0 


hy ou, hy Our h3 0u3 : 


> Show that for orthogonal curvilinear coordinates Vu; = @;/h;. Hence show that the two 
sets of vectors {@} and {é;} are identical in this case. 


Letting ® = u; in (10.60) we find immediately that Vu; = 6;/h;. Therefore |Vu;| = 1/h;, and 
so &; = Vu;/|Vuil = hiVu; = &. < 


Divergence 
In order to derive the expression for the divergence of a vector field in orthogonal 
curvilinear coordinates, we must first write the vector field in terms of the basis 
vectors of the coordinate system: 
a = a, e + dz @ + a3 63. 
The divergence is then given by 


at | 2 a psaiy 2 aan) danas) 
hyhyh; | du, eas Ou are 0u3 or (10.61) 
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> Prove the expression for V -a in orthogonal curvilinear coordinates. 


Let us consider the sub-expression V - (a;é,). Now @; = € x @3 = )Vup X h3Vu3. Therefore 
V - (a1€1) = V- (aihoh3Vu2 x Vus), 
= V(ahzh3) - (Vu2 x Vu3) + arhoh3V - (Vu2 x Vu). 
However, V - (Vu2 x Vu3) = 0, from (10.43), so we obtain 


Ms é é 
V = (aie) = V(arhohy) (F x 2) V(aihzhs) « ae a 
letting ® = a,hyhz in (10.60) and substituting into the above equation, we find 
1 3 
“(aye hyl 
V-(a1@1) = his au =—(arhzhs). 


Repeating the analysis for V-(a)@) and V- (a363), and adding the results we obtain (10.61), 
as required. < 
Laplacian 

In the expression for the divergence (10.61), let 

100, 1 Oo , 1 0® 

a=V®= ——@, —— &;, 

hy Ou, sl hy 0 aie. a h3 0u3 7 

where we have used (10.60). We then obtain 


1 0 [hyh; 0® 0 [hgh od 0 [(hyhy 0® 

y) 2113 3H] 11) 

O= 

v hyhyh3 E ( hy =) = Our ( hy x) x 0u3 ( h3 =) é 


which is the expression for the Laplacian in orthogonal curvilinear coordinates. 


Curl 
The curl of a vector field a = ay@; + an€@) + a3@3 in orthogonal curvilinear 
coordinates is given by 

hye; = In@ N36 


ee |e (10.62) 


hyhyhs Ou, Ou Ou 


hia, hoa h3a3 


> Prove the expression for V x a in orthogonal curvilinear coordinates. 


Let us consider the sub-expression V x (a;é,). Since €, = h; Vu, we have 
Vx (ae) =Vx (ah, Vu;), 
= Viayhy) x Vuy + ayhV x Vuy. 


But V x Vu; = 0, so we obtain 


Vx (a,@;) = V(ayh,) x ~. 
1 
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1 0o®, 1 6®@, 1 6® 


odo = —— —— ——é 
¥ hy Ou, ctor hy Our xe h3 0u3 3 
1 [ 0 0 ) 
: = —— |}/—(hyh —(h3h —(hyh 
V-a ies aia 2h3a,) + Dan 3hya2) + Ban! 1 ae 


Wye, hay 3&3 
1 0 é 0 


V = — — 
ie iiighs| Ou Sin Ou 
hia, hoa, h3a, 
1 [| 6 (hyhy 6® 0 (h3h, 0® 0 (hyh, 0® 
2.2) aa ee a, eee ef 
¥ hyhyh; | uy ( hy Ou, ane hy Ou ae h; 0u3 


Table 10.4 Vector operators in orthogonal curvilinear coordinates uy, U2, U3. 
® is a scalar field and a is a vector field. 


Letting ® = a;h; in (10.60) and garages into the above equation, we find 


é 0 
| 
ih (a thy) — ia Bun —(a,h;). 
The corresponding analysis of V x (a2 @2) produces terms in @; and @,, whilst that of 
V x (a3@3) produces terms in @, and @). When the three results are added together, the 
coefficients multiplying €;, @, and @; are the same as those obtained by writing out (10.62) 
explicitly, thus proving the stated result. < 


V x (a1@)) = 


The general expressions for the vector operators in orthogonal curvilinear 
coordinates are shown for reference in table 10.4. The explicit results for cylindrical 
and spherical polar coordinates, given in tables 10.2 and 10.3 respectively, are 
obtained by substituting the appropriate set of scale factors in each case. 

A discussion of the expressions for vector operators in tensor form, which 
are valid even for non-orthogonal curvilinear coordinate systems, is given in 
chapter 26. 


10.11 Exercises 
10.1 Evaluate the integral 
/ [a(b- a +b-a) + a(b- a) — 2(a- a)b— blal”] de 
in which a, b are the derivatives of a, b with respect to t. 


10.2 At time t = 0, the vectors E and B are given by E = Ep and B = Bo, where the 
unit vectors, Ej and Bo are fixed and orthogonal. The equations of motion are 


dE 
—=E Bx E 
oF 0 + B x Eo, 
dB 
—=B E x Bo. 
Th 0 +E x Bo 


Find E and B at a general time t, showing that after a long time the directions 
of E and B have almost interchanged. 
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10.3 


10.4 


10.5 


The general equation of motion of a (non-relativistic) particle of mass m and 
charge q when it is placed in a region where there is a magnetic field B and an 
electric field E is 


mi = q(E + £ xB); 


here r is the position of the particle at time t and r = dr/dt, etc. Write this as 
three separate equations in terms of the Cartesian components of the vectors 
involved. 

For the simple case of crossed uniform fields E = Ei, B = Bj, in which the 
particle starts from the origin at t = 0 with rt = vok, find the equations of motion 
and show the following: 


(a) if v) = E/B then the particle continues its initial motion; 
(b) if vo = 0 then the particle follows the space curve given in terms of the 
parameter € by 
mE mE 
——(1—cosé), = 0, z= ——(é-siné). 
Bag | ie i Big (¢ é) 
Interpret this curve geometrically and relate € to t. Show that the total 
distance travelled by the particle after time t is given by 
2E f' Bat’ 
B Jo 2m 
Use vector methods to find the maximum angle to the horizontal at which a stone 
may be thrown so as to ensure that it is always moving away from the thrower. 
If two systems of coordinates with a common origin O are rotating with respect 
to each other, the measured accelerations differ in the two systems. Denoting 


by r and r’ position vectors in frames OX YZ and OX'Y'Z’, respectively, the 
connection between the two is 


x= 


dt’. 


¥ =f+oxr+20xit+o x (@ xn), 


where is the angular velocity vector of the rotation of OX YZ with respect to 
OX'Y’Z’ (taken as fixed). The third term on the RHS is known as the Coriolis 
acceleration, whilst the final term gives rise to a centrifugal force. 

Consider the application of this result to the firing of a shell of mass m from 
a stationary ship on the steadily rotating earth, working to the first order in 
@ (= 7.3 x 10~> rads“). If the shell is fired with velocity v at time t = 0 and only 
reaches a height that is small compared with the radius of the earth, show that 
its acceleration, as recorded on the ship, is given approximately by 


¥ = g— 20 x (v+ 8), 
where mg is the weight of the shell measured on the ship’s deck. 


The shell is fired at another stationary ship (a distance s away) and v is such 
that the shell would have hit its target had there been no Coriolis effect. 


(a) Show that without the Coriolis effect the time of flight of the shell would 
have been t = —2g - v/g”. 

(b) Show further that when the shell actually hits the sea it is off-target by 
approximately 


2 1 
aie xX @)- V](gt + v) — (@ X vr? — 710 x g)t?. 
(c) Estimate the order of magnitude A of this miss for a shell for which the 
initial speed v is 300 ms“, firing close to its maximum range (v makes an 


angle of 2/4 with the vertical) in a northerly direction, whilst the ship is 
stationed at latitude 45° North. 
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10.6 


10.7 


10.8 


10.9 


10.10 


10.11 


Prove that for a space curve r = r(s), where s is the arc length measured along 
the curve from a fixed point, the triple scalar product 


dr . @r\ ar 
ds” ds?) ds3 
at any point on the curve has the value xt, where x is the curvature and t the 


torsion at that point. 
For the twisted space curve y* + 27axz — 8la’y = 0, given parametrically by 


x = au(3—1W), y = 3a’, z=au(34+u), 
show that the following hold: 


(a) ds/du= 3,/2a(1 +u’), where s is the distance along the curve measured from 
the origin; 

(b) the length of the curve from the origin to the Cartesian point (2a, 3a, 4a) is 
4,/2a; 

(c) the radius of curvature at the point with parameter u is 3a(1 + v7); 

(d) the torsion t and curvature « at a general point are equal; 

(e) any of the Frenet—-Serret formulae that you have not already used directly 
are satisfied. 


The shape of the curving slip road joining two motorways, that cross at right 
angles and are at vertical heights z = 0 and z = h, can be approximated by the 
space curve 


ZT\ , /2h So FLIX. 

ay) it * Insin (5) i+ 2k 

Show that the radius of curvature p of the slip road is (2h/1)cosec (zm/h) at 
height z and that the torsion t = —1/p. To shorten the algebra, set z = 2h0/x 
and use 0 as the parameter. 

In a magnetic field, field lines are curves to which the magnetic induction B is 
everywhere tangential. By evaluating dB/ds, where s is the distance measured 
along a field line, prove that the radius of curvature at any point on a line is 
given by 


r 


v2h Incos ( 
T 


B 


P= BxB- VB. 


Find the areas of the given surfaces using parametric coordinates. 


(a) Using the parameterisation x = ucos@¢, y = using, z = ucotQ, find the 
sloping surface area of a right circular cone of semi-angle Q whose base has 
radius a. Verify that it is equal to + xperimeter of the base xslope height. 

(b) Using the same parameterization as in (a) for x and y, and an appropriate 
choice for z, find the surface area between the planes z = 0 and z = Z of 
the paraboloid of revolution z = a(x? + y’). 


Parameterising the hyperboloid 


e'RP 2 
by x = acos O@sec ¢, y = bsin@ sec d, z = ctan@, show that an area element on 
its surface is 


dS = sec’ ¢ [c? sec? @ (b* cos” 0 + a? sin’ 0) + a’? tan? | Ne 98 do. 
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10.12 


10.13 


10.14 


10.15 


10.16 


10.17 


10.18 


Use this formula to show that the area of the curved surface x? + y?—2?7 = a@ 
between the planes z = 0 and z = 2a is 


na (s + S sinh! 24/3) : 


For the function 
—x2_y2 


2(x,y) = (x? — ye ; 


find the location(s) at which the steepest gradient occurs. What are the magnitude 
and direction of that gradient? The algebra involved is easier if plane polar 
coordinates are used. 

Verify by direct calculation that 


V-(ax b) =b-(V x a)—a-(V xb). 
In the following exercises, a, b and ¢ are vector fields. 
(a) Simplify 
Vxa(V-a) + ax[Vx(Vxa)] + ax Va. 
(b) By explicitly writing out the terms in Cartesian coordinates, prove that 
[c-(b- V)—b-(c-V)]a=(V x a): (bx ¢). 
(c) Prove that a x (V x a) = V(5a’) —(a- V)a. 


Evaluate the Laplacian of the function 


2x? 


x2 nS y? Es 22 

(a) directly in Cartesian coordinates, and (b) after changing to a spherical polar 
coordinate system. Verify that, as they must, the two methods give the same 
result. 

Verify that (10.42) is valid for each component separately when a is the Cartesian 
vector x*yi+xyzj+z’yk, by showing that each side of the equation is equal to 
zi+(2x+2z)j+xk. 

The (Maxwell) relationship between a time-independent magnetic field B and the 
current density J (measured in SI units in Am~’) producing it, 


V x B= WJ, 


~O% YZ) = 


can be applied to a long cylinder of conducting ionised gas which, in cylindrical 
polar coordinates, occupies the region p < a. 


(a) Show that a uniform current density (0,C,0) and a magnetic field (0,0, B), 
with B constant (= Bo) for p > a and B = B(p) for p < a, are consistent 
with this equation. Given that B(0) = 0 and that B is continuous at p = a, 
obtain expressions for C and B(p) in terms of Bo and a. 

(b) The magnetic field can be expressed as B = V x A, where A is known as 
the vector potential. Show that a suitable A that has only one non-vanishing 
component, Ay(p), can be found, and obtain explicit expressions for A4(p) 
for both p < a and p > a. Like B, the vector potential is continuous at p = a. 

(c) The gas pressure p(p) satisfies the hydrostatic equation Vp = J x B and 
vanishes at the outer wall of the cylinder. Find a general expression for p. 


Evaluate the Laplacian of a vector field using two different coordinate systems 
as follows. 
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10.11 EXERCISES 


10.19 


10.20 


(a) For cylindrical polar coordinates p, @,z, evaluate the derivatives of the three 
unit vectors with respect to each of the coordinates, showing that only 06, /d¢ 
and 0€,/0 are non-zero. 


(i) Hence evaluate V*a when a is the vector 6,, i.e. a vector of unit magnitude 
everywhere directed radially outwards and expressed by a, = 1, ay = 
a, =0. 

(ii) Note that it is trivially obvious that V x a = 0 and hence that equation 
(10.41) requires that V(V - a) = Va. 

(iii) Evaluate V(V - a) and show that the latter equation holds, but that 


[VV -a)], # V2a,. 


(b) Rework the same problem in Cartesian coordinates (where, as it happens, 
the algebra is more complicated). 


Maxwell’s equations for electromagnetism in free space (i.e. in the absence of 
charges, currents and dielectric or magnetic media) can be written 


(i) V-B=0, (ii) V-E=0, 
(iii) VxE+S =0, jy Ue Sy 


A vector A is defined by B= V x A, and a scalar ¢ by E = —V¢ — 0A/dt. Show 
that if the condition 
10d _ 5 


is imposed (this is known as choosing the Lorentz gauge), then A and ¢ satisfy 
wave equations as follows: 


1& 
2 
Oe nae 
1A 
2 
(vii) VAS ao = 


The reader is invited to proceed as follows. 


(a) Verify that the expressions for B and E in terms of A and ¢ are consistent 
with (i) and (iii). 

(b) Substitute for E in (ii) and use the derivative with respect to time of (v) to 
eliminate A from the resulting expression. Hence obtain (vi). 

(c) Substitute for B and E in (iv) in terms of A and @. Then use the gradient of 
(v) to simplify the resulting equation and so obtain (vii). 


In a description of the flow of a very viscous fluid that uses spherical polar 
coordinates with axial symmetry, the components of the velocity field u are given 
in terms of the stream function yp by 


1 op - —1 ody 
sind 00 
Find an explicit expression for the differential operator E defined by 


Ey =—(rsin0)(V X u)g. 


rsin@ ér° 


The stream function satisfies the equation of motion E*p = 0 and, for the flow of 
a fluid past a sphere, takes the form y(r,0) = f(r) sin? 0. Show that f(r) satisfies 
the (ordinary) differential equation 


if —4r’f" + 8rf’ — 8f =0. 
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10.21 


10.22 


10.23 


10.24 


Paraboloidal coordinates u,v, @ are defined in terms of Cartesian coordinates by 
x = uvcos ¢, y =uvsind, 7 3(u — 0"). 


Identify the coordinate surfaces in the u,v, @ system. Verify that each coordinate 
surface (u = constant, say) intersects every coordinate surface on which one of 
the other two coordinates (v, say) is constant. Show further that the system of 
coordinates is an orthogonal one and determine its scale factors. Prove that the 
u-component of V x a is given by 


1 ap 4 Ody 1 Ga, 
(u2+v?)!/2 \ iv év uv 0- 
Non-orthogonal curvilinear coordinates are difficult to work with and should be 
avoided if at all possible, but the following example is provided to illustrate the 
content of section 10.10. 

In a new coordinate system for the region of space in which the Cartesian 
coordinate z satisfies z > 0, the position of a point r is given by (a1, %, R), where 
a, and a are respectively the cosines of the angles made by r with the x- and y- 
coordinate axes of a Cartesian system and R = |r|. The ranges are —1 < g; < 1, 
0<R<o@. 


(a) Express r in terms of «1, #2, R and the unit Cartesian vectors i, j, k. 
(b) Obtain expressions for the vectors e; (= Or/0a1,...) and hence show that the 
scale factors h; are given by 
RU = 3)? RU =a)? 
> 2 > 

(1 — a? — 0f)1/2 (1— a? — of)1/2 
Verify formally that the system is not an orthogonal one. 
Show that the volume element of the coordinate system is 

R do, day dR 
(a ay 
and demonstrate that this is always less than or equal to the corresponding 
expression for an orthogonal curvilinear system. 


(e) Calculate the expression for (ds)? for the system, and show that it differs 
from that for the corresponding orthogonal system by 
20; 4 R* 


1 2 = w doyday. 
1 °2 


hy h3 = 1. 


ws 
ee 


dV = 


Hyperbolic coordinates u,v, are defined in terms of Cartesian coordinates by 
x =coshucosvcos ¢, y =coshucosv sin ¢, z =sinhusinv. 


Sketch the coordinate curves in the ¢ = 0 plane, showing that far from the origin 
they become concentric circles and radial lines. In particular, identify the curves 
u=0,v =0, v = 2/2 and v = a. Calculate the tangent vectors at a general 
point, show that they are mutually orthogonal and deduce that the appropriate 
scale factors are 


1/2 
> 


a h,= (cosh? u— cos? v) hy = cosh ucosv. 


Find the most general function p(u) of u only that satisfies Laplace’s equation 
Vp =0. 

In a Cartesian system, A and B are the points (0,0, —1) and (0,0, 1) respectively. 
In a new coordinate system a general point P is given by (u,uU2,u3) with 
uy = ("1 +12), i = 3(r1 —1), u; = d; here r; and ry are the distances AP and 
BP and ¢ is the angle between the plane ABP and y = 0. 
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10.12 HINTS AND ANSWERS 


10.1 


10.3 


10.5 


10.7 


10.9 


10.11 
10.13 


10.15 


10.17 


10.19 


10.21 


(a) Express z and the perpendicular distance p from P to the z-axis in terms of 
Uy, U2, U3. 

(b) Evaluate 0x/0u;, 0y/Ou;, 0z/0u;, for i = 1, 2, 3. 

(c) Find the Cartesian components of ti; and hence show that the new coordi- 
nates are mutually orthogonal. Evaluate the scale factors and the infinitesimal 
volume element in the new coordinate system. 

(d) Determine and sketch the forms of the surfaces u; = constant. 

(e) Find the most general function f of only that satisfies V7f = 0. 


10.12 Hints and answers 


Group the term so that they form the total derivatives of compound vector 
expressions. The integral has the value a x (a x b) +h. 

For crossed uniform fields, +(Bq/m)’x = q(E—Bvo)/m, j = 0, mz = qBx+mvg; 
(b) € = Bqt/m; the path is a cycloid in the plane y = 0; ds = [(dx/dt)? + 
(dz/dt)]'/" dt. 

g =r —@ x (m Xr), where is the shell’s acceleration measured by an observer 
fixed in space. To first order in @, the direction of g is radial, i.e. parallel to i’. 


(a) Note that s is orthogonal to g. 
(b) If the actual time of flight is T, use (s + A) - g = 0 to show that 
T x u(1+2¢-7(g x w)-v+---). 
In the Coriolis terms, it is sufficient to put T = t. 
(c) For this situation (g x w)-v=0 and w x v=0;1 43s and A= 10-15 m 
to the East. 
(a) Evaluate (dr/du) - (dr/du). 
(b) Integrate the previous result between u = 0 and u = 1. 
(c) t= [Y20 +u2)} [0 — wi + 2uj + (1 + Jk]. Use dt/ds = (dt/du)/(ds/du); 
p! = |dt/ds|. 
(d) f= (1 +02)! [-2ui+ (1 —1?)j]. b = [21 + u?)) 7 [(w? — 1)i— 2uj + (1 +.u)k). 
Use db/ds = (db/du)/(ds/du) and show that this equals —[3a(1 + w2)?]-'A. 
(ce) Show that da/ds = t(b — t) = —2[3./2a(1 + w?)3]-! [(1 — u?)i + uj]. 
Note that dB = (dr- V)B and that B = Bt, with t= dr/ds. Obtain (B- V)B/B = 
t(dB/ds) + fi(B/p) and then take the vector product of t with this equation. 
To integrate sec” @(sec” @ + tan? )'/? dd put tan d = 27!” sinh y. 
Work in Cartesian coordinates, regrouping the terms obtained by evaluating the 
divergence on the LHS. 
(a) 2z(x2+-y?2+22)-3[(y?2 +22)(y?2-+22—3x2)—4x4] ; (b) 2r7! cos 0 (1—5 sin? 6 cos? ); 
both are equal to 2zr~4(r? — 5x’). 
Use the formulae given in table 10.2. 


(a) C = —Bo/(Hoa); B(p) = Bop/a. 

(b) Bop?/(3a) for p <a, and Bo[p/2 — a*/(6p)] for p > a. 

(c) [Bo/(2H0)I[1 — (p/a)]. 

Recall that V x V¢ = 0 for any scalar @ and that 6/ét and V act on different 
variables. 

Two sets of paraboloids of revolution about the z-axis and the sheaf of planes 
containing the z-axis. For constant u,—oo < z < u?/2; for constant v,—v?/2 < 
z <oo. The scale factors are h, = h, = (wu? +07)", hg = ww. 
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10.23 The tangent vectors are as follows: for u = 0, the line joining (1,0,0) and 
(—1,0,0); for v = 0, the line joining (1,0,0) and (00,0,0); for v = 2/2, the line 
(0,0, 2); for v = z, the line joining (—1,0,0) and (—oo, 0,0). 
w(u) = 2tan7! e” +c, derived from é[cosh u(dp/du)]/du = 0. 
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Line, surface and volume integrals 


In the previous chapter we encountered continuously varying scalar and vector 
fields and discussed the action of various differential operators on them. In 
addition to these differential operations, the need often arises to consider the 
integration of field quantities along lines, over surfaces and throughout volumes. 
In general the integrand may be scalar or vector in nature, but the evaluation 
of such integrals involves their reduction to one or more scalar integrals, which 
are then evaluated. In the case of surface and volume integrals this requires the 
evaluation of double and triple integrals (see chapter 6). 


11.1 Line integrals 


In this section we discuss line or path integrals, in which some quantity related 
to the field is integrated between two given points in space, A and B, along a 
prescribed curve C that joins them. In general, we may encounter line integrals 


of the forms 
[eae faa fe x dr, (11.1) 
Cc Cc Cc 


where ¢ is a scalar field and a is a vector field. The three integrals themselves are 
respectively vector, scalar and vector in nature. As we will see below, in physical 
applications line integrals of the second type are by far the most common. 

The formal definition of a line integral closely follows that of ordinary integrals 
and can be considered as the limit of a sum. We may divide the path C joining 
the points A and B into N small line elements Ar,, p = 1,...,N. If (Xp, Yp, Zp) is 
any point on the line element Ar, then the second type of line integral in (11.1), 
for example, is defined as 


N 
[a -dr= jim » A(Xp, Vp, Zp) * At, 


where it is assumed that all |Ar,| > 0 as N > oo. 
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Each of the line integrals in (11.1) is evaluated over some curve C that may be 
either open (A and B being distinct points) or closed (the curve C forms a loop, 
so that A and B are coincident). In the case where C is closed, the line integral 
is written ¢. to indicate this. The curve may be given either parametrically by 
r(u) = x(u)i+ y(uw)j + z(u)k or by means of simultaneous equations relating x, y,z 
for the given path (in Cartesian coordinates). A full discussion of the different 
representations of space curves was given in section 10.3. 

In general, the value of the line integral depends not only on the end-points 
A and B but also on the path C joining them. For a closed curve we must also 
specify the direction around the loop in which the integral is taken. It is usually 
taken to be such that a person walking around the loop C in this direction 
always has the region R on his/her left; this is equivalent to traversing C in the 
anticlockwise direction (as viewed from above). 


11.1.1 Evaluating line integrals 


The method of evaluating a line integral is to reduce it to a set of scalar integrals. 
It is usual to work in Cartesian coordinates, in which case dr = dxi+dyj+dzk. 
The first type of line integral in (11.1) then becomes simply 


dr =i ,y,z)d j ,y,z)dy +k ,y,z)dz. 
[oa if dosnzrds+i [ dosszrdy+ [ obser: 


The three integrals on the RHS are ordinary scalar integrals that can be evaluated 
in the usual way once the path of integration C has been specified. Note that in 
the above we have used relations of the form 


J diax =i [ ax 


which is allowable since the Cartesian unit vectors are of constant magnitude 
and direction and hence may be taken out of the integral. If we had been using 
a different coordinate system, such as spherical polars, then, as we saw in the 
previous chapter, the unit basis vectors would not be constant. In that case the 
basis vectors could not be factorised out of the integral. 

The second and third line integrals in (11.1) can also be reduced to a set of 
scalar integrals by writing the vector field a in terms of its Cartesian components 
as a = dyi+ a,j + a_k, where a,, a,, a, are each (in general) functions of x, y, z. 
The second line integral in (11.1), for example, can then be written as 


[adem [Cait asi + ack) (dvit dy + dehy 
Cc Cc 
= [ia dx + ay dy + a, dz) 
c 


= facdst fady+ fade. (11.2) 
c Cc Cc 
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A similar procedure may be followed for the third type of line integral in (11.1), 
which involves a cross product. 

Line integrals have properties that are analogous to those of ordinary integrals. 
In particular, the following are useful properties (which we illustrate using the 
second form of line integral in (11.1) but which are valid for all three types). 


(i) Reversing the path of integration changes the sign of the integral. If the 
path C along which the line integrals are evaluated has A and B as its 


end-points then 
B A 
| aar=— a- dr. 
A B 


This implies that if the path C is a loop then integrating around the loop 
in the opposite direction changes the sign of the integral. 

(ii) If the path of integration is subdivided into smaller segments then the sum 
of the separate line integrals along each segment is equal to the line integral 
along the whole path. So, if P is any point on the path of integration that 
lies between the path’s end-points A and B then 


B P B 
| adr= | adr+ | a- dr. 
A A P 


> Evaluate the line integral I = {.a- dr, where a = (x + y)it+(y —x)j, along each of the 
paths in the xy-plane shown in figure 11.1, namely 


(i) the parabola y? = x from (1,1) to (4,2), 


(ii) the curve x = 2u*+u+1, y=1+1* from (1,1) to (4,2), 
(iii) the line y = 1 from (1,1) to (4,1), followed by the line x = 4 from (4,1) 
to (4,2). 


Since each of the paths lies entirely in the xy-plane, we have dr = dxi+dyj. We can 
therefore write the line integral as 


t= facdr= [t+yax+(—s ay. (11.3) 


We must now evaluate this line integral along each of the prescribed paths. 
Case (i). Along the parabola y? = x we have 2y dy = dx. Substituting for x in (11.3) 
and using just the limits on y, we obtain 


(4,2) 2 
r= f [xt y)dv ty — xd] = [ (2 +y)2y +9 — dy = 114. 
(1) 1 


Note that we could just as easily have substituted for y and obtained an integral in x, 
which would have given the same result. 

Case (ii). The second path is given in terms of a parameter u. We could eliminate u 
between the two equations to obtain a relationship between x and y directly and proceed 
as above, but it is usually quicker to write the line integral in terms of the parameter u. 
Along the curve x = 2u?+u+1, y= 1+? we have dx = (4u+ 1)du and dy = 2udu. 
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Figure 11.1 Different possible paths between the points (1,1) and (4, 2). 


Substituting for x and y in (11.3) and writing the correct limits on u, we obtain 


(4,2) 
f -| (eee yas GS) a) 
(1,1) 


1 
= i) [(3u? + u + 2)(4u + 1) — (a? + u)2u] du = 103. 
0 


Case (iii). For the third path the line integral must be evaluated along the two line 
segments separately and the results added together. First, along the line y = 1 we have 
dy = 0. Substituting this into (11.3) and using just the limits on x for this segment, we 
obtain 


(4) 4 
: [ix + y)dx + (y—a)dy] = f (e+ Ide = 105, 
(11) 1 


Next, along the line x = 4 we have dx = 0. Substituting this into (11.3) and using just the 
limits on y for this segment, we obtain 


(4,2) 2 
[ tetndxty say = fw —4ydy = 24. 
(4,1) 1 


The value of the line integral along the whole path is just the sum of the values of the line 
integrals along each segment, and is given by I = 10; — 2} =8. 

When calculating a line integral along some curve C, which is given in terms 
of x, y and z, we are sometimes faced with the problem that the curve C is such 
that x, y and z are not single-valued functions of one another over the entire 
length of the curve. This is a particular problem for closed loops in the xy-plane 
(and also for some open curves). In such cases the path may be subdivided into 
shorter line segments along which one coordinate is a single-valued function of 
the other two. The sum of the line integrals along these segments is then equal 
to the line integral along the entire curve C. A better solution, however, is to 
represent the curve in a parametric form r(u) that is valid for its entire length. 
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> Evaluate the line integral I = ¢..xdy, where C is the circle in the xy-plane defined by 


aL SF, 2 =O) 


Adopting the usual convention mentioned above, the circle C is to be traversed in the 
anticlockwise direction. Taking the circle as a whole means x is not a single-valued 
function of y. We must therefore divide the path into two parts with x = +,/a? — y? for 
the semicircle lying to the right of x = 0, and x = —,/a? — y? for the semicircle lying to 
the left of x = 0. The required line integral is then the sum of the integrals along the two 
semicircles. Substituting for x, it is given by 


—a 
a 


I f xay [ Ve Fay+ f (-Ve=7) dy 
Cc —a 
a4 [lea Fay = ne 
0 


Alternatively, we can represent the entire circle parametrically, in terms of the azimuthal 
angle ¢, so that x = acos¢ and y = asing with ¢ running from 0 to 2z. The integral can 
therefore be evaluated over the whole circle at once. Noting that dy = acos dd, we can 
rewrite the line integral completely in terms of the parameter ¢ and obtain 


2 


t= xdy=o [ cos’ pdb = na’. < 
c 0 


11.1.2 Physical examples of line integrals 


There are many physical examples of line integrals, but perhaps the most common 
is the expression for the total work done by a force F when it moves its point 
of application from a point A to a point B along a given curve C. We allow the 
magnitude and direction of F to vary along the curve. Let the force act at a point 
r and consider a small displacement dr along the curve; then the small amount 
of work done is dW = F - dr, as discussed in subsection 7.6.1 (note that dW can 
be either positive or negative). Therefore, the total work done in traversing the 


path C is 
We = | Fear, 
Cc 


Naturally, other physical quantities can be expressed in such a way. For example, 
the electrostatic potential energy gained by moving a charge q along a path C in 
an electric field E is —q [, E- dr. We may also note that Ampére’s law concerning 
the magnetic field B associated with a current-carrying wire can be written as 


f B-de= pol, 
Cc 


where I is the current enclosed by a closed path C traversed in a right-handed 
sense with respect to the current direction. 
Magnetostatics also provides a physical example of the third type of line 
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integral in (11.1). If a loop of wire C carrying a current I is placed in a magnetic 
field B then the force dF on a small length dr of the wire is given by dF = I dr xB, 
and so the total (vector) force on the loop is 


F=1 4 dexB, 
Cc 


11.1.3 Line integrals with respect to a scalar 


In addition to those listed in (11.1), we can form other types of line integral, 
which depend on a particular curve C but for which we integrate with respect 
to a scalar du, rather than the vector differential dr. This distinction is somewhat 
arbitrary, however, since we can always rewrite line integrals containing the vector 
differential dr as a line integral with respect to some scalar parameter. If the path 
C along which the integral is taken is described parametrically by r(u) then 


dr 
= —du 
du” 


and the second type of line integral in (11.1), for example, can be written as 


dr 
[acam [a Fa 


A similar procedure can be followed for the other types of line integral in (11.1). 
Commonly occurring special cases of line integrals with respect to a scalar are 


| o ds, | ads, 
Cc Cc 


where s is the arc length along the curve C. We can always represent C paramet- 
rically by r(u), and from section 10.3 we have 


dr 


The line integrals can therefore be expressed entirely in terms of the parameter u 
and thence evaluated. 


> Evaluate the line integral I = fe(x—yy ds, where C is the semicircle of radius a running 
from A = (a,0) to B = (—a,0) and for which y > 0. 


The semicircular path from A to B can be described in terms of the azimuthal angle @ 
(measured from the x-axis) by 


r(d) = acos di+asin dj, 


where ¢ runs from 0 to z. Therefore the element of arc length is given, from section 10.3, 
by 


ds = ,| — - —dd =a(cos’ + sin’ 6) db = add. 
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a &. 


(a) (b) (c) 


Figure 11.2 (a) A simply connected region; (b) a doubly connected region; 
(c) a triply connected region. 


Since (x — y)? = a?(1 — sin 2¢), the line integral becomes 


2 * 3 : Jno 
I [os y) ds [ «a sin2)d¢ = na’. 4 


As discussed in the previous chapter, the expression (10.58) for the square of 
the element of arc length in three-dimensional orthogonal curvilinear coordinates 
Uy, U2, U3 is 


(ds)? = hi (duy)? +h} (din + A} (dsy’, 


where hy, hy, h3 are the scale factors of the coordinate system. If a curve C in 
three dimensions is given parametrically by the equations u; = u;(A) for i = 1,2,3 
then the element of arc length along the curve is 


11.2 Connectivity of regions 


In physical systems it is usual to define a scalar or vector field in some region R. 
In the next and some later sections we will need the concept of the connectivity 
of such a region in both two and three dimensions. 

We begin by discussing planar regions. A plane region R is said to be simply 
connected if every simple closed curve within R can be continuously shrunk to 
a point without leaving the region (see figure 11.2(a)). If, however, the region 
R contains a hole then there exist simple closed curves that cannot be shrunk 
to a point without leaving R (see figure 11.2(b)). Such a region is said to be 
doubly connected, since its boundary has two distinct parts. Similarly, a region 
with n — 1 holes is said to be n-fold connected, or multiply connected (the region 
in figure 11.2(c) is triply connected). 
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Figure 11.3. A simply connected region R bounded by the curve C. 


These ideas can be extended to regions that are not planar, such as general 
three-dimensional surfaces and volumes. The same criteria concerning the shrink- 
ing of closed curves to a point also apply when deciding the connectivity of such 
regions. In these cases, however, the curves must lie in the surface or volume 
in question. For example, the interior of a torus is not simply connected, since 
there exist closed curves in the interior that cannot be shrunk to a point without 
leaving the torus. The region between two concentric spheres of different radii is 
simply connected. 


11.3 Green’s theorem in a plane 


In subsection 11.1.1 we considered (amongst other things) the evaluation of line 
integrals for which the path C is closed and lies entirely in the xy-plane. Since 
the path is closed it will enclose a region R of the plane. We now discuss how to 
express the line integral around the loop as a double integral over the enclosed 
region R. 

Suppose the functions P(x, y), Q(x, y) and their partial derivatives are single- 
valued, finite and continuous inside and on the boundary C of some simply 
connected region R in the xy-plane. Green’s theorem in a plane (sometimes called 
the divergence theorem in two dimensions) then states 


fr dx+Qdy)= Md (2 - =) dx dy, (11.4) 


and so relates the line integral around C to a double integral over the enclosed 
region R. This theorem may be proved straightforwardly in the following way. 
Consider the simply connected region R in figure 11.3, and let y = y,(x) and 
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y = yo(x) be the equations of the curves STU and SVU respectively. We then 


write 
oP b yolx) gp y=yr(x) 
ae a ay =f dsl Phe 
|. oy ies / «f : oy / | . Dov 
b 
=f [Posvats)) — Pls nies)) ax 
b a 
=— f Posmcode— f Poayats)de =— f P de. 
a b Cc 


If we now let x = x1(y) and x = x2(y) be the equations of the curves TSV and 
TUV respectively, we can similarly show that 


ae ara a 
= [' [020.9 ac10.»)] ay 


d 
-/ Osnvdy+ | Osnvdy= f Ody. 


Subtracting these two results gives Green’s theorem in a plane. 


> Show that the area of a region R enclosed by a simple closed curve C is given by A = 
3 fol a) ydx) = $.xdy =— $. y dx. Hence calculate the area of the ellipse x = acos ¢, 
y =bsin®g. 


In Green’s theorem (11.4) put P = —y and Q = x; then 


fisay ydx) ik + 1) dx dy 2 ff dx dy = 2A. 


Therefore the area of the region is A = ; fox dy —y dx). Alternatively, we could put P = 0 
and Q = x and obtain A = ¢. xdy, or put P = —y and Q =0, which gives A = — f, ydx. 
The area of the ellipse x = acos ¢, y = bsin¢ is given by 


2n 
A 5 f (xdy ydx) = a ab(cos’ o + sin? ») dp 
2Ic 2 Io 


2n 
= ab db = nab. < 
2 Jo 


It may further be shown that Green’s theorem in a plane is also valid for 
multiply connected regions. In this case, the line integral must be taken over 
all the distinct boundaries of the region. Furthermore, each boundary must be 
traversed in the positive direction, so that a person travelling along it in this 
direction always has the region R on their left. In order to apply Green’s theorem 
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CQ 


Xx 


Figure 11.4 A doubly connected region R bounded by the curves C; and C). 


to the region R shown in figure 11.4, the line integrals must be taken over 
both boundaries, C; and C, in the directions indicated, and the results added 
together. 

We may also use Green’s theorem in a plane to investigate the path indepen- 
dence (or not) of line integrals when the paths lie in the xy-plane. Let us consider 
the line integral 


B 
I -| (P dx + Qdy). 


For the line integral from A to B to be independent of the path taken, it must 
have the same value along any two arbitrary paths C; and C? joining the points. 
Moreover, if we consider as the path the closed loop C formed by C; — C, then 
the line integral around this loop must be zero. From Green’s theorem in a plane, 
(11.4), we see that a sufficient condition for I = 0 is that 


Eee (11.5) 


fa) 
oy Ox 


throughout some simply connected region R containing the loop, where we assume 
that these partial derivatives are continuous in R. 

It may be shown that (11.5) is also a necessary condition for I = 0 and is 
equivalent to requiring P dx + Qdy to be an exact differential of some function 
(x, y) such that P dx + Q dy = dg. It follows that fre dx+Q dy) = ¢(B)— (A) 
and that ¢.(P dx + Qdy) around any closed loop C in the region R is identically 
zero. These results are special cases of the general results for paths in three 
dimensions, which are discussed in the next section. 
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> Evaluate the line integral 


I -¢ [(e*y + cos x sin y) dx + (e* + sinx cos y) dy], 
c 


around the ellipse x/a* + y*/b? = 1. 


Clearly, it is not straightforward to calculate this line integral directly. However, if we let 
P=e*‘y+cosxsiny and Q =e*+sinxcos y, 


then 0P/dy = e* + cosxcosy = 0Q/0x, and so P dx + Qdy is an exact differential (it 
is actually the differential of the function f(x,y) = e*y + sinxsiny). From the above 
discussion, we can conclude immediately that J = 0. < 


11.4 Conservative fields and potentials 


So far we have made the point that, in general, the value of a line integral 
between two points A and B depends on the path C taken from A to B. In the 
previous section, however, we saw that, for paths in the xy-plane, line integrals 
whose integrands have certain properties are independent of the path taken. We 
now extend that discussion to the full three-dimensional case. 

For line integrals of the form fa: dr, there exists a class of vector fields for 
which the line integral between two points is independent of the path taken. Such 
vector fields are called conservative. A vector field a that has continuous partial 
derivatives in a simply connected region R is conservative if, and only if, any of 
the following is true. 


(i) The integral ee a-dr, where A and B lie in the region R, is independent of 
the path from A to B. Hence the integral ¢.a- dr around any closed loop 
in R is zero. 

(ii) There exists a single-valued function ¢ of position such that a = V¢. 

(iii) Vxa=0. 
(iv) a- dr is an exact differential. 


The validity or otherwise of any of these statements implies the same for the 
other three, as we will now show. 

First, let us assume that (i) above is true. If the line integral from A to B 
is independent of the path taken between the points then its value must be a 
function only of the positions of A and B. We may therefore write 


i a- dr = ¢(B)— o(A), (11.6) 
A 


which defines a single-valued scalar function of position @. If the points A and B 
are separated by an infinitesimal displacement dr then (11.6) becomes 


a-dr = dd, 
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which shows that we require a-dr to be an exact differential: condition (iv). From 
(10.27) we can write db = V@ - dr, and so we have 


(a—V¢)-dr =0. 


Since dr is arbitrary, we find that a = Vd; this immediately implies V x a = 0, 
condition (iii) (see (10.37)). 

Alternatively, if we suppose that there exists a single-valued function of position 
@ such that a = V¢ then V x a = 0 follows as before. The line integral around a 
closed loop then becomes 


facd=$vo-de= $ dé. 


Since we defined ¢ to be single-valued, this integral is zero as required. 

Now suppose V x a = 0. From Stoke’s theorem, which is discussed in sec- 
tion 11.9, we immediately obtain ¢. a-dr = 0; then a = V¢ and a-dr = d¢ follow 
as above. 

Finally, let us suppose a- dr = dd. Then immediately we have a = V@, and the 
other results follow as above. 


> Evaluate the line integral I = ale a- dr, where a = (xy? +z)i+ (x?y + 2)j+ xk, A is the 
point (c,c,h) and B is the point (2c,c/2,h), along the different paths 


(i) Ci, given by x =cu, y=c/u,z=h, 
(ii) C2, given by 2y = 3c—x,z=h. 


Show that the vector field a is in fact conservative, and find @ such that a=V¢. 


Expanding out the integrand, we have 
(2c, ¢/2,h) 
t= fo [oy? +2)dx + (2y +2) dy +d], (11.7) 
(c,c,h) 


which we must evaluate along each of the paths C; and C). 
(i) Along C, we have dx = cdu, dy = —(c/u’) du, dz = 0, and on substituting in (11.7) 
and finding the limits on u, we obtain 


g 2 
I [el =) du = c(h— 1). 


(ii) Along C) we have 2dy = —dx, dz = 0 and, on substituting in (11.7) and using the 
limits on x, we obtain 


2c 
r= | (42° — Sex? + 3e?x + h—1) dx = c(h— 1). 


Hence the line integral has the same value along paths C; and C. Taking the curl of a, 
we have 


V xa=(0—0)i+ (1 —1)j + Qxy — 2xy)k = 0, 


so a is a conservative vector field, and the line integral between two points must be 
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independent of the path taken. Since a is conservative, we can write a = V¢. Therefore, ¢ 
must satisfy 


cy 2 
BE = xy +2, 


which implies that ¢ = 3x? y? +2zx+f(y,z) for some function f. Secondly, we require 
0b of 
oy oy 
which implies f = 2y + g(z). Finally, since 
0 0 


=xy¢ HP axty 42, 


we have g = constant = k. It can be seen that we have explicitly constructed the function 
b= hry text2y+k.< 

The quantity @ that figures so prominently in this section is called the scalar 
potential function of the conservative vector field a (which satisfies V x a = 0), and 
is unique up to an arbitrary additive constant. Scalar potentials that are multi- 
valued functions of position (but in simple ways) are also of value in describing 
some physical situations, the most obvious example being the scalar magnetic 
potential associated with a current-carrying wire. When the integral of a field 
quantity around a closed loop is considered, provided the loop does not enclose 
a net current, the potential is single-valued and all the above results still hold. If 
the loop does enclose a net current, however, our analysis is no longer valid and 
extra care must be taken. 

If, instead of being conservative, a vector field b satisfies V-b = 0 (ie. b 
is solenoidal) then it is both possible and useful, for example in the theory of 
electromagnetism, to define a vector field a such that b = V x a. It may be shown 
that such a vector field a always exists. Further, if a is one such vector field then 
a’ =a+Vy-+c, where yp is any scalar function and ¢ is any constant vector, also 
satisfies the above relationship, i.e. b = V x a’. This was discussed more fully in 
subsection 10.8.2. 


11.5 Surface integrals 


As with line integrals, integrals over surfaces can involve vector and scalar fields 
and, equally, can result in either a vector or a scalar. The simplest case involves 
entirely scalars and is of the form 


[oas. (11.8) 
S 


As analogues of the line integrals listed in (11.1), we may also encounter surface 
integrals involving vectors, namely 


[oes [aa [a x dS. (11.9) 
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(a) (b) 


Figure 11.5 (a) A closed surface and (b) an open surface. In each case a 
normal to the surface is shown: dS = ndS. 


All the above integrals are taken over some surface S, which may be either 
open or closed, and are therefore, in general, double integrals. Following the 
notation for line integrals, for surface integrals over a closed surface f¥ is replaced 
by ¢,. 

The vector differential dS in (11.9) represents a vector area element of the 
surface S. It may also be written dS = fidS, where fi is a unit normal to the 
surface at the position of the element and dS is the scalar area of the element used 
in (11.8). The convention for the direction of the normal fi to a surface depends 
on whether the surface is open or closed. A closed surface, see figure 11.5(a), 
does not have to be simply connected (for example, the surface of a torus is not), 
but it does have to enclose a volume V, which may be of infinite extent. The 
direction of fi is taken to point outwards from the enclosed volume as shown. 
An open surface, see figure 11.5(b), spans some perimeter curve C. The direction 
of fi is then given by the right-hand sense with respect to the direction in which 
the perimeter is traversed, ie. follows the right-hand screw rule discussed in 
subsection 7.6.2. An open surface does not have to be simply connected but for 
our purposes it must be two-sided (a MObius strip is an example of a one-sided 
surface). 

The formal definition of a surface integral is very similar to that of a line 
integral. We divide the surface S into N elements of area AS,, p = 1,2,...,N, 
each with a unit normal ftp. If (xp, yp, Zp) is any point in AS, then the second type 
of surface integral in (11.9), for example, is defined as 


N 
| a-dS= lim Zane YpsZp) * fipAS), 


where it is required that all AS, + 0 as N > o. 
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dA 
a 


Figure 11.6 A surface S (or part thereof) projected onto a region R in the 
xy-plane; dS is a surface element. 


11.5.1 Evaluating surface integrals 


We now consider how to evaluate surface integrals over some general surface. This 
involves writing the scalar area element dS in terms of the coordinate differentials 
of our chosen coordinate system. In some particularly simple cases this is very 
straightforward. For example, if S is the surface of a sphere of radius a (or some 
part thereof) then using spherical polar coordinates 0, on the sphere we have 
dS = a’ sin@d0 dd. For a general surface, however, it is not usually possible to 
represent the surface in a simple way in any particular coordinate system. In such 
cases, it is usual to work in Cartesian coordinates and consider the projections of 
the surface onto the coordinate planes. 

Consider a surface (or part of a surface) S as in figure 11.6. The surface S is 
projected onto a region R of the xy-plane, so that an element of surface area dS 
projects onto the area element dA. From the figure, we see that dA = |cosa| dS, 
where « is the angle between the unit vector k in the z-direction and the unit 
normal fi to the surface at P. So, at any given point of S, we have simply 


_ dA _ dA 
~ |cosa|  |fi- kl’ 


Now, if the surface S is given by the equation f(x, y,z) = 0 then, as shown in sub- 
section 10.7.1, the unit normal at any point of the surface is given by f = Vf /|Vf| 
evaluated at that point, cf. (10.32). The scalar element of surface area then becomes 


dA |Vf\dA__ |Vf|dA 


dS = Vf-k  of/oz’ 


(11.10) 
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where |Vf| and éf/0z are evaluated on the surface S. We can therefore express 
any surface integral over S as a double integral over the region R in the xy-plane. 


> Evaluate the surface integral I = {,a-dS, where a = xi and S is the surface of the 


hemisphere x* + y* +2? =a? with z > 0. 


The surface of the hemisphere is shown in figure 11.7. In this case dS may be easily 
expressed in spherical polar coordinates as dS = a? sin 0 d0 dd, and the unit normal to the 
surface at any point is simply f. On the surface of the hemisphere we have x = asin @ cos 
and so 


a- dS = x(i-f) dS = (asin cos $)(sin 0 cos @)(a’ sin 0 dO dd). 


Therefore, inserting the correct limits on 0 and ¢, we have 


n/2 
t= faas=a [ awsina fo do cos? 4 =e 
Ss 0 


We could, however, follow the general prescription above and project the hemisphere S 
onto the region R in the xy-plane that is a circle of radius a centred at the origin. Writing 
the equation of the surface of the hemisphere as f(x, y) = x7 + y? +z? —a? =0 and using 


(11.10), we have 
7 _ 7 i) Wilda 
t= fads [ x@-#) as = [x (i- ) OF faz” 


Now Vf = 2xi+ 2yj + 2zk = 2r, so on the surface S we have |Vf| = 2|r| = 2a. On S we 
also have Of /0z = 2z = 2\/a? — x* — y? and i- f = x/a. Therefore, the integral becomes 


x2 
t= [| > ecay, 
r f@—x—y? y 


Although this integral may be evaluated directly, it is quicker to transform to plane polar 
coordinates: 


2n 3 dp 
= cos’ fd ee ae 
E ee 0 Va —p? 


Making the substitution = asinu, we finally obtain 


2n n/2 2 3 
rf cos? de | a@sin'udu=—“. « 
0 0 3 


In the above discussion we assumed that any line parallel to the z-axis intersects 
S only once. If this is not the case, we must split up the surface into smaller 
surfaces S;, Sp etc. that are of this type. The surface integral over S is then the 
sum of the surface integrals over S;, S; and so on. This is always necessary for 
closed surfaces. 

Sometimes we may need to project a surface S (or some part of it) onto the 
zx- or yz-plane, rather than the xy-plane; for such cases, the above analysis is 
easily modified. 
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Figure 11.7 The surface of the hemisphere x? + y? +z? = a’, z > 0. 


11.5.2 Vector areas of surfaces 


The vector area of a surface S is defined as 


s= | as, 
Ss 


where the surface integral may be evaluated as above. 


> Find the vector area of the surface of the hemisphere x* + y* +z? = a with z > 0. 


As in the previous example, dS = a’ sin 0 dO d@f in spherical polar coordinates. Therefore 


the vector area is given by 
S= [[esino 0 do. 
s 


Now, since f varies over the surface S, it also must be integrated. This is most easily 
achieved by writing f in terms of the constant Cartesian basis vectors. On S we have 


f = sin@cos di+sin@ sin dj + cos 0k, 


so the expression for the vector area becomes 


Qn n/2 Qn n/2 
S=i @ | cos bd | sin? 0d0) +j «| sin odo | sin’ 0 d0 
0 0 0 0 
2a n/2 
+k (*/ ap [ sinoox odo) 
0 0 


=04+04 2a°k = na’k. 


Note that the magnitude of S is the projected area, of the hemisphere onto the xy-plane, 
and not the surface area of the hemisphere. < 
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Figure 11.8 The conical surface spanning the perimeter C and having its 
vertex at the origin. 


The hemispherical shell discussed above is an example of an open surface. For 
a closed surface, however, the vector area is always zero. This may be seen by 
projecting the surface down onto each Cartesian coordinate plane in turn. For 
each projection, every positive element of area on the upper surface is cancelled 
by the corresponding negative element on the lower surface. Therefore, each 
component of S = f, dS vanishes. 

An important corollary of this result is that the vector area of an open surface 
depends only on its perimeter, or boundary curve, C. This may be proved as 
follows. If surfaces S, and S, have the same perimeter then S; — S) is a closed 


surface, for which 
fas= | as— | dS =0. 
5, S> 


Hence S; = S:. Moreover, we may derive an expression for the vector area of 
an open surface S solely in terms of a line integral around its perimeter C. 
Since we may choose any surface with perimeter C, we will consider a cone 
with its vertex at the origin (see figure 11.8). The vector area of the elementary 
triangular region shown in the figure is dS = zr x dr. Therefore, the vector area 
of the cone, and hence of any open surface with perimeter C, is given by the line 


integral 
S= sf r x dr. 
2Ic 


For a surface confined to the xy-plane, r = xi+ yj and dr = dxi+ dyj, 
and we obtain for this special case that the area of the surface is given by 
A= 5 f(x dy — ydx), as we found in section 11.3. 
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> Find the vector area of the surface of the hemisphere x* + y? +2? = a’, z > 0, by 


evaluating the line integral S = 3 Fx dr around its perimeter. 


The perimeter C of the hemisphere is the circle x? + y* = a’, on which we have 
r=acosdi+asingj,  dr=—asindddi+acos dddj. 
Therefore the cross product r x dr is given by 
i j k 
acos asingd 0 
—asinddp acosddd 0 


rx dr= =a’(cos’ @ + sin’ 6) dbk = a dbk, 


and the vector area becomes 


2n 
S= srk | dd =na’k. < 


11.5.3 Physical examples of surface integrals 


There are many examples of surface integrals in the physical sciences. Surface 
integrals of the form (11.8) occur in computing the total electric charge on a 
surface or the mass of a shell, f, p(r) dS, given the charge or mass density p(r). 
For surface integrals involving vectors, the second form in (11.9) is the most 
common. For a vector field a, the surface integral f,a-dS is called the flux 
of a through S$. Examples of physically important flux integrals are numerous. 
For example, let us consider a surface S in a fluid with density p(r) that has a 
velocity field v(r). The mass of fluid crossing an element of surface area dS in 
time dt is dM = pv- dSdt. Therefore the net total mass flux of fluid crossing S 
is M = f, p(r)v(r) - dS. As a another example, the electromagnetic flux of energy 
out of a given volume V bounded by a surface S is ,(E x H)- dS. 

The solid angle, to be defined below, subtended at a point O by a surface (closed 
or otherwise) can also be represented by an integral of this form, although it is 
not strictly a flux integral (unless we imagine isotropic rays radiating from O). 


The integral 
g/t ea/. (11.11) 
sf Soot 


gives the solid angle Q subtended at O by a surface S if r is the position vector 
measured from O of an element of the surface. A little thought will show that 
(11.11) takes account of all three relevant factors: the size of the element of 
surface, its inclination to the line joining the element to O and the distance from 


O. Such a general expression is often useful for computing solid angles when the 
three-dimensional geometry is complicated. Note that (11.11) remains valid when 
the surface S is not convex and when a single ray from O in certain directions 
would cut S in more than one place (but we exclude multiply connected regions). 
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In particular, when the surface is closed Q = 0 if O is outside S and Q = 4z if O 
is an interior point. 

Surface integrals resulting in vectors occur less frequently. An example is 
afforded, however, by the total resultant force experienced by a body immersed in 
a stationary fluid in which the hydrostatic pressure is given by p(r). The pressure 
is everywhere inwardly directed and the resultant force is F = — ¢, pdS, taken 
over the whole surface. 


11.6 Volume integrals 


Volume integrals are defined in an obvious way and are generally simpler than 
line or surface integrals since the element of volume dV is a scalar quantity. We 
may encounter volume integrals of the forms 


[ oar. [ aay. (11.12) 


Clearly, the first form results in a scalar, whereas the second form yields a vector. 
Two closely related physical examples, one of each kind, are provided by the total 
mass of a fluid contained in a volume V, given by f,, p(r) dV, and the total linear 
momentum of that same fluid, given by f,, p(r)v(r)dV where v(r) is the velocity 
field in the fluid. As a slightly more complicated example of a volume integral we 
may consider the following. 


> Find an expression for the angular momentum of a solid body rotating with angular 
velocity @ about an axis through the origin. 


Consider a small volume element dV situated at position r; its linear momentum is p dV¥, 
where p = p(r) is the density distribution, and its angular momentum about O is r x prdV. 
Thus for the whole body the angular momentum L is 


L= fe x bpdv. 


Putting t = @ xr yields 


_ = 2 — . 
L= | ex(oxlpav [oe pdV fee o)rpdV. <4 


The evaluation of the first type of volume integral in (11.12) has already been 
considered in our discussion of multiple integrals in chapter 6. The evaluation of 
the second type of volume integral follows directly since we can write 


faa aifaavsif aaven [ aay, (11.13) 
Vv V V Vv 


where ay, dy, a, are the Cartesian components of a. Of course, we could have 
written a in terms of the basis vectors of some other coordinate system (e.g. 
spherical polars) but, since such basis vectors are not, in general, constant, they 
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Figure 11.9 A general volume V containing the origin and bounded by the 
closed surface S. 


cannot be taken out of the integral sign as in (11.13) and must be included as 
part of the integrand. 


11.6.1 Volumes of three-dimensional regions 


As discussed in chapter 6, the volume of a three-dimensional region V is simply 
V = f, dV, which may be evaluated directly once the limits of integration have 
been found. However, the volume of the region obviously depends only on the 
surface S that bounds it. We should therefore be able to express the volume V 
in terms of a surface integral over S. This is indeed possible, and the appropriate 
expression may derived as follows. Referring to figure 11.9, let us suppose that 
the origin O is contained within V. The volume of the small shaded cone is 
dV = ar - dS; the total volume of the region is thus given by 


1 


It may be shown that this expression is valid even when O is not contained in V. 
Although this surface integral form is available, in practice it is usually simpler 
to evaluate the volume integral directly. 


> Find the volume enclosed between a sphere of radius a centred on the origin and a circular 
cone of half-angle « with its vertex at the origin. 


The element of vector area dS on the surface of the sphere is given in spherical polar 
coordinates by a’ sin 0 d0 df ¥. Now taking the axis of the cone to lie along the z-axis (from 
which @ is measured) the required volume is given by 


2n Oo 
vazfrasq> / as | a’ sindr-d0 
3 Js 3 Jo 0 


2n OL 3 
-;/ as | @aaeide Cheetah 
cae A 3 
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11.7 Integral forms for grad, div and curl 


In the previous chapter we defined the vector operators grad, div and curl in purely 
mathematical terms, which depended on the coordinate system in which they were 
expressed. An interesting application of line, surface and volume integrals is the 
expression of grad, div and curl in coordinate-free, geometrical terms. If ¢ is a 
scalar field and a is a vector field then it may be shown that at any point P 


. 1 

Vo= lim. (Ff as) (11.14) 
. 1 

V-a= lim (F faa) (11.15) 
; 1 

Vxa= lim (Ff axa) (11.16) 


where V is a small volume enclosing P and S is its bounding surface. Indeed, 
we may consider these equations as the (geometrical) definitions of grad, div and 
curl. An alternative, but equivalent, geometrical definition of V x a at a point P, 
which is often easier to use than (11.16), is given by 


es 1 
(Vx a)-= lim (5 f adr), (11.17) 


where C is a plane contour of area A enclosing the point P and fi is the unit 
normal to the enclosed planar area. 

It may be shown, in any coordinate system, that all the above equations are 
consistent with our definitions in the previous chapter, although the difficulty of 
proof depends on the chosen coordinate system. The most general coordinate 
system encountered in that chapter was one with orthogonal curvilinear coordi- 
nates 1, U2,u3, of which Cartesians, cylindrical polars and spherical polars are all 
special cases. Although it may be shown that (11.14) leads to the usual expression 
for grad in curvilinear coordinates, the proof requires complicated manipulations 
of the derivatives of the basis vectors with respect to the coordinates and is not 
presented here. In Cartesian coordinates, however, the proof is quite simple. 


> Show that the geometrical definition of grad leads to the usual expression for Vd in 
Cartesian coordinates. 


Consider the surface S of a small rectangular volume element AV = Ax Ay Az that has its 
faces parallel to the x, y, and z coordinate surfaces; the point P (see above) is at one corner. 
We must calculate the surface integral (11.14) over each of its six faces. Remembering that 
the normal to the surface points outwards from the volume on each face, the two faces 
with x = constant have areas AS = —iAy Az and AS = iAy Az respectively. Furthermore, 
over each small surface element, we may take ¢ to be constant, so that the net contribution 
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to the surface integral from these two faces is then 


[( + Ad) — Ay Azi = ( x6) ey Rei 
= Fax dy dz. 


The surface integral over the pairs of faces with y = constant and z = constant respectively 
may be found in a similar way, and we obtain 


ab. , ad . 
foas- (Bi itZitg k) Axavac 


Therefore V@ at the point P is given by 


= = 1 0d, , 0b, oo, 
eels \e isa ite aes 
_ op, ah, eK 
~ Ox Vay) I+3 Oz 


We now turn to (11.15) and (11.17). These geometrical definitions may be 
shown straightforwardly to lead to the usual expressions for div and curl in 
orthogonal curvilinear coordinates. 


> By considering the infinitesimal volume element dV = hyhzh3 Au; Auz Auz shown in fig- 
ure 11.10, show that (11.15) leads to the usual expression for V-a in orthogonal curvilinear 
coordinates. 


Let us write the vector field in terms of its components with respect to the basis vectors 
of the curvilinear coordinate system as a = ayjé; + d2@) + a3@3. We consider first the 
contribution to the RHS of (11.15) from the two faces with u; = constant, ie. PORS 
and the face opposite it (see figure 11.10). Now, the volume element is formed from the 
orthogonal vectors h, Au; @;, hy Auy é and h3 Au; @3 at the point P and so for PORS we 
have 

AS = hyh3 Aur Au3 6 x & => —hyh; Aur Au3 e. 


Reasoning along the same lines as in the previous example, we conclude that the contri- 
bution to the surface integral of a- dS over PQRS and its opposite face taken together is 
given by 
0 0 
—(a AS) Auy = —(ayhzh3) Au, Aun Au3. 
Ou, Ou, 
The surface integrals over the pairs of faces with u. = constant and u3; = constant 


respectively may be found in a similar way, and we obtain 


fa see | ahh aa ka | Mia hs 
S ou, Our 0u3 


Therefore V-a at the point P is given by 


1 
‘a=. li ———— .dS 
Ye inj Aitp Aug 30 F hyh; Au, Au, Au; fa 


1 r) 
5 (ash) .< 
0u3 


é 
lbs EE (ayhyh3) + Buy 2h) 
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ere 


hy Aur, 


x 


Figure 11.10 A general volume AV in orthogonal curvilinear coordinates 
U1,U2,U3. PT gives the vector h, Au; @;, PS gives h, Au,@ and PQ gives 
h Au3 é3. 


The planar surface PORS is defined by the orthogonal vectors hy Auz@, and h; Au; 3 
at the point P. If we traverse the loop in the direction PSRQ then, by the right-hand 
convention, the unit normal to the plane is @,. Writing a = a,@, + a@, + 363, the line 
integral around the loop in this direction is given by 


A 
$ a-dr azh Aur t ass + e (ash) | Au3 
PSRO un 
= aah + (aah) | Aur 2 a3h3 Au; 
0u3 


é i) 
= | lost) — ata)] An As 
Our 0u3 
Therefore from (11.17) the component of V x a in the direction @, at P is given by 
1 
= ii —_——_. -d 
w - a) Aun 50 lia Au Au3 b vr a | 
1 0 é 
= hyh, E (h3as) 0u3 (ra) . 


The other two components are found by cyclically permuting the subscripts 1, 2, 3. < 


Finally, we note that we can also write the V? operator as a surface integral by 
setting a = V@ in (11.15), to obtain 


v=V-vé= lim (7 f V6-as). 
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11.8 Divergence theorem and related theorems 


The divergence theorem relates the total flux of a vector field out of a closed 
surface S to the integral of the divergence of the vector field over the enclosed 
volume V; it follows almost immediately from our geometrical definition of 
divergence (11.15). 

Imagine a volume V, in which a vector field a is continuous and differentiable, 
to be divided up into a large number of small volumes V;. Using (11.15), we have 
for each small volume 


where S; is the surface of the small volume V;. Summing over i we find that 
contributions from surface elements interior to S cancel since each surface element 
appears in two terms with opposite signs, the outward normals in the two terms 
being equal and opposite. Only contributions from surface elements that are also 
parts of S survive. If each V; is allowed to tend to zero then we obtain the 
divergence theorem, 


[vad = paas (11.18) 
Vv S 


We note that the divergence theorem holds for both simply and multiply con- 
nected surfaces, provided that they are closed and enclose some non-zero volume 
V. The divergence theorem may also be extended to tensor fields (see chapter 26). 
The theorem finds most use as a tool in formal manipulations, but sometimes 
it is of value in transforming surface integrals of the form [,a-dS into volume 
integrals or vice versa. For example, setting a =r we immediately obtain 


[vray = [ sav =3v = br-ds, 
Vv Vv s 


which gives the expression for the volume of a region found in subsection 11.6.1. 
The use of the divergence theorem is further illustrated in the following example. 


> Evaluate the surface integral I = [,a- dS, where a= (y—x)i+ x?zjt+(z+x?)k and S 
is the open surface of the hemisphere x? + y* +27 =a’, z > 0. 


We could evaluate this surface integral directly, but the algebra is somewhat lengthy. We 
will therefore evaluate it by use of the divergence theorem. Since the latter only holds 
for closed surfaces enclosing a non-zero volume JV, let us first consider the closed surface 
S’ = § + S;, where S; is the circular area in the xy-plane given by x7 + y? < a’, z =0; S’ 
then encloses a hemispherical volume V. By the divergence theorem we have 


[vad =f ads= fa-as+ [ a- dS. 
Vv ' s 5, 


Now V.a=—1+0+1=0, so we can write 


[acs=— [ acas, 
s S, 
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x 


Figure 11.11 A closed curve C in the xy-plane bounding a region R. Vectors 
tangent and normal to the curve at a given point are also shown. 


The surface integral over S; is easily evaluated. Remembering that the normal to the 
surface points outward from the volume, a surface element on S, is simply dS = —k dx dy. 
On S, we also have a = (y — x)i+x7k, so that 


1r=-f ads = [fs axdy, 
Sy R 


where R is the circular region in the xy-plane given by x*+y? < a’. Transforming to plane 
polar coordinates we have 


2n a nat 
r= ff pcos’ & pdpdg = [ cos? de | p>dp = —. < 
R 0 0 4 


It is also interesting to consider the two-dimensional version of the divergence 
theorem. As an example, let us consider a two-dimensional planar region R in 
the xy-plane bounded by some closed curve C (see figure 11.11). At any point 
on the curve the vector dr = dxi+dyj is a tangent to the curve and the vector 
fids = dyi—dxj is a normal pointing out of the region R. If the vector field a is 
continuous and differentiable in R then the two-dimensional divergence theorem 
in Cartesian coordinates gives 


) 
II (S t =) dxdy = fads =f (acdy— ay dx) 
r\ Ox oy Cc : 


Letting P = —a, and Q = a,, we recover Green’s theorem in a plane, which was 
discussed in section 11.3. 


11.8.1 Green’s theorems 


Consider two scalar functions ¢ and w that are continuous and differentiable in 
some volume V bounded by a surface S. Applying the divergence theorem to the 
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vector field (Vw we obtain 
f ovy-ds= | V-(ovyav 
Ss V 
-{ [dV>p + (Vo) -(Vy)] dV. (11.19) 


Reversing the roles of ¢ and p in (11.19) and subtracting the two equations gives 


$ (¢Vp — pV): dS = ‘i (PV-p — pV’) dV. (11.20) 
S Vv 


Equation (11.19) is usually known as Green’s first theorem and (11.20) as his 
second. Green’s second theorem is useful in the development of the Green’s 
functions used in the solution of partial differential equations (see chapter 21). 


11.8.2 Other related integral theorems 


There exist two other integral theorems which are closely related to the divergence 
theorem and which are of some use in physical applications. If @ is a scalar field 
and b is a vector field and both ¢ and b satisfy our usual differentiability 
conditions in some volume V bounded by a closed surface S then 


[ voar = $ ois (11.21) 
V x bdV = $ dSxb. (11.22) 
jay 


> Use the divergence theorem to prove (11.21). 


In the divergence theorem (11.18) let a = ¢c, where ¢ is a constant vector. We then have 


[v-oar = f eas 


Expanding out the integrand on the LHS we have 
V-(¢c) = oV-c+e-Vd=c-Vd, 


since ¢ is constant. Also, 6c -dS =c- dS, so we obtain 


[-(Wonav = fe-oas 


Since ¢ is constant we may take it out of both integrals to give 


c: | voav =e. f pas, 


and since ¢ is arbitrary we obtain the stated result (11.21). < 


Equation (11.22) may be proved in a similar way by letting a = b x c in the 
divergence theorem, where c is again a constant vector. 
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11.8.3 Physical applications of the divergence theorem 


The divergence theorem is useful in deriving many of the most important partial 
differential equations in physics (see chapter 20). The basic idea is to use the 
divergence theorem to convert an integral form, often derived from observation, 
into an equivalent differential form (used in theoretical statements). 


>For a compressible fluid with time-varying position-dependent density p(r,t) and velocity 
field v(r,t), in which fluid is neither being created nor destroyed, show that 


op 
Tye a As 


For an arbitrary volume V in the fluid, the conservation of mass tells us that the rate of 
increase or decrease of the mass M of fluid in the volume must equal the net rate at which 
fluid is entering or leaving the volume, ie. 


dM 
sien ta 


where S is the surface bounding V. But the mass of fluid in V is simply M = [,, pdV, so 


we have 
d 
5, [ pay + f pv-ds=0. 


Taking the derivative inside the first integral on the RHS and using the divergence theorem 
to rewrite the second integral, we obtain 


A A 
[ Sars [v-womar = [ oP + Vv -(py)| dV =0. 
y Ot Vv y | ot 


Since the volume V is arbitrary, the integrand (which is assumed continuous) must be 
identically zero, so we obtain 


op 
ot 
This is known as the continuity equation. It can also be applied to other systems, for 
example those in which p is the density of electric charge or the heat content, etc. For the 


flow of an incompressible fluid, p = constant and the continuity equation becomes simply 
V-v=0.< 


+V- (py) =0. 


In the previous example, we assumed that there were no sources or sinks in 
the volume V, ie. that there was no part of V in which fluid was being created 
or destroyed. We now consider the case where a finite number of point sources 
and/or sinks are present in an incompressible fluid. Let us first consider the 
simple case where a single source is located at the origin, out of which a quantity 
of fluid flows radially at a rate Q (m? s~!). The velocity field is given by 


Qr Of 
v= — = — >. 
4nr3 4? 
Now, for a sphere S; of radius r centred on the source, the flux across S; is 


f v- dS = |v|4ar? = O. 


Si 
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Since v has a singularity at the origin it is not differentiable there, i.e. V-v is not 
defined there, but at all other points V- v = 0, as required for an incompressible 
fluid. Therefore, from the divergence theorem, for any closed surface S$) that does 
not enclose the origin we have 


¢v-as= | v-vav =o. 
S> Vv 


Thus we see that the surface integral ¢,v-dS has value Q or zero depending 
on whether or not S encloses the source. In order that the divergence theorem is 
valid for all surfaces S, irrespective of whether they enclose the source, we write 


V-v=0Q0(r), 


where 6(r) is the three-dimensional Dirac delta function. The properties of this 
function are discussed fully in chapter 13, but for the moment we note that it is 
defined in such a way that 


d(r—a)=0 forr #a, 


. f(a) ifa lies in V 
d(r—a)dV = 
- f(r)d(r — a) dV 


0 otherwise 


for any well-behaved function f(r). Therefore, for any volume V containing the 
source at the origin, we have 


[v-vav =0 | amar =o. 


which is consistent with ¢,v- dS = Q for a closed surface enclosing the source. 
Hence, by introducing the Dirac delta function the divergence theorem can be 
made valid even for non-differentiable point sources. 

The generalisation to several sources and sinks is straightforward. For example, 
if a source is located at r = a and a sink at r = b then the velocity field is 


y_ tt=ao — (r—b)0 


~ Anijr—als 4xir—bP 


and its divergence is given by 


V-v= Qo(r —a) — Qo(r — b). 


Therefore, the integral ¢,v-dS has the value Q if S encloses the source, —Q if 
S encloses the sink and 0 if S encloses neither the source nor sink or encloses 
them both. This analysis also applies to other physical systems — for example, in 
electrostatics we can regard the sources and sinks as positive and negative point 
charges respectively and replace v by the electric field E. 
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11.9 Stokes’ theorem and related theorems 


Stokes’ theorem is the ‘curl analogue’ of the divergence theorem and relates the 
integral of the curl of a vector field over an open surface S to the line integral of 
the vector field around the perimeter C bounding the surface. 

Following the same lines as for the derivation of the divergence theorem, we 
can divide the surface S into many small areas S; with boundaries C; and unit 
normals hj. Using (11.17), we have for each small area 


(V Xx a)- fi six 4 a-dr. 
Ci 
Summing over i we find that on the RHS all parts of all interior boundaries 
that are not part of C are included twice, being traversed in opposite directions 
on each occasion and thus contributing nothing. Only contributions from line 
elements that are also parts of C survive. If each S; is allowed to tend to zero 
then we obtain Stokes’ theorem, 


Vv -dS= - dr. 11.23 
[ik x a) fa dr. ( ) 


We note that Stokes’ theorem holds for both simply and multiply connected open 
surfaces, provided that they are two-sided. Stokes’ theorem may also be extended 
to tensor fields (see chapter 26). 

Just as the divergence theorem (11.18) can be used to relate volume and surface 
integrals for certain types of integrand, Stokes’ theorem can be used in evaluating 
surface integrals of the form ¢,(V x a) - dS as line integrals or vice versa. 


Given the vector field a = yi—xj+7 zk, verify Stokes’ theorem for the hemispherical 


surface x? ++y?+27=a?,z>0. 


Let us first evaluate the surface integral 


[0rxaras 


over the hemisphere. It is easily shown that V x a = —2k, and the surface element is 
dS = a’ sin 6 dé d@ f in spherical polar coordinates. Therefore 


2n n/2 
[oxa-as= | ap [ d0 (—2a? sin 0) #-k 
AY 0 0 


2n n/2 
22.579 ing (2 
227; 1 ap [ sind (=) d0 
2n n/2 
= 2a | dp ‘i sin 0 cos 6 d0 = —2na’. 
0 0 


We now evaluate the line integral around the perimeter curve C of the surface, which 
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is the circle x* + y? =a’ in the xy-plane. This is given by 


fade =p (yim xj+ zk): (dxit dyj+ deh) 
Gc c 


= fly dx xd) 
c 


Using plane polar coordinates, on C we have x = acos¢, y = asing so that dx = 
—asin ddd, dy = acos ¢ dg, and the line integral becomes 


2n 2n 
guy dx —xdy)= -« | (sin? @ + cos? @) do = -« | dé = —2na’. 
c 0 0 


Since the surface and line integrals have the same value, we have verified Stokes’ theorem 
in this case. < 


The two-dimensional version of Stokes’ theorem also yields Green’s theorem in 
a plane. Consider the region R in the xy-plane shown in figure 11.11, in which a 
vector field a is defined. Since a = a, i+a,j, we have V x a = (0a, /0x—0a,/dy)k, 
and Stokes’ theorem becomes 


day =) ¢ 
— —— } dxdy= a, dx + a, dy). 
a ( ax dy y A y dy) 


Letting P = a, and Q = ay we recover Green’s theorem in a plane, (11.4). 


11.9.1 Related integral theorems 


As for the divergence theorem, there exist two other integral theorems that are 
closely related to Stokes’ theorem. If ¢ is a scalar field and b is a vector field, 
and both ¢ and b satisfy our usual differentiability conditions on some two-sided 
open surface S bounded by a closed perimeter curve C, then 


Jas x Ve -¢ o dr, (11.24) 
Ss Cc 
[usxvixb= fdr xb, (11.25) 
Ss Cc 


> Use Stokes’ theorem to prove (11.24). 


In Stokes’ theorem, (11.23), let a = ce, where ¢ is a constant vector. We then have 


| [V x (de)] -dS = ¢ oe: dr. (11.26) 
s c 
Expanding out the integrand on the LHS we have 

V x (gc) = Vd xe+oV xc=Vod xe, 


since ¢ is constant, and the scalar triple product on the LHS of (11.26) can therefore be 
written 


[V x (de)] - dS = (Vp x c)- dS =e: (dS x V9). 
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Substituting this into (11.26) and taking ¢ out of both integrals because it is constant, we 


find 
e- [asxvene- f par. 
s c 


Since ¢ is an arbitrary constant vector we therefore obtain the stated result (11.24). < 


Equation (11.25) may be proved in a similar way, by letting a = b xc in Stokes’ 
theorem, where c is again a constant vector. We also note that by setting b =r 


in (11.25) we find 
[isxvxr= p drxe. 
s c 


Expanding out the integrand on the LHS gives 
(dS x V) x r= dS— dS(V -r) = dS — 3dS = —2dS. 


Therefore, as we found in subsection 11.5.2, the vector area of an open surface S 


is given by 
1 
s= [as=5 fp rxar 
s Cc 


11.9.2 Physical applications of Stokes’ theorem 


Like the divergence theorem, Stokes’ theorem is useful in converting integral 
equations into differential equations. 


>From Ampére’s law, derive Maxwell’s equation in the case where the currents are steady, 
ie. VX B= oJ = 0. 


Ampere’s rule for a distributed current with current density J is 


f Bde=ww [ 3-as, 
Cc s 


for any circuit C bounding a surface S. Using Stokes’ theorem, the LHS can be transformed 
into {,(V x B)- dS; hence 


[00x B= 1d) ts =0 
Ss 


for any surface S. This can only be so if V x B— oJ = 0, which is the required relation. 
Similarly, from Faraday’s law of electromagnetic induction we can derive Maxwell’s 
equation V x E = —éB/ét. < 

In subsection 11.8.3 we discussed the flow of an incompressible fluid in the 
presence of several sources and sinks. Let us now consider vortex flow in an 
incompressible fluid with a velocity field 


Ta 
v= —e9, 


in cylindrical polar coordinates p, ¢, z. For this velocity field V x v equals zero 
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everywhere except on the axis p = 0, where v has a singularity. Therefore ¢. v- dr 
equals zero for any path C that does not enclose the vortex line on the axis and 
2x if C does enclose the axis. In order for Stokes’ theorem to be valid for all 
paths C, we therefore set 


V x v= 220(p), 


where 6(p) is the Dirac delta function, to be discussed in subsection 13.1.3. Now, 
since V x v = 0, except on the axis p = 0, there exists a scalar potential y such 
that v = Vp. It may easily be shown that y = @, the polar angle. Therefore, if C 
does not enclose the axis then 


fvdr= fap=o, 


and if C does enclose the axis, 


fi v-dr= Ag = 2an, 
c 


where n is the number of times we traverse C. Thus ¢ is a multivalued potential. 

Similar analyses are valid for other physical systems — for example, in magneto- 
statics we may replace the vortex lines by current-carrying wires and the velocity 
field v by the magnetic field B. 


11.10 Exercises 
11.1 The vector field F is defined by 
F = 2xzi + 2yz7j + (x? + 2y?z — Ik. 


Calculate V x F and deduce that F can be written F = Vd. Determine the form 
of ¢. 
11.2 The vector field Q is defined by 


Q= BPrOt2a+yV42] i+ BYZty4t 242°] [4+ BLO+y +P 4+y']k 


Show that Q is a conservative field, construct its potential function and hence 
evaluate the integral J = {Q- dr along any line connecting the point A at 
(1,—1,1) to B at (2,1,2). 

11.3 F is a vector field xy*i+2j+ xk, and L is a path parameterised by x = ct, y = c/t, 
z =d for the range 1 <1 < 2. Evaluate (a) f, Fdt, (b) [, Fdy and (c) f, F- dr. 

11.4 By making an appropriate choice for the functions P(x, y) and Q(x, y) that appear 
in Green’s theorem in a plane, show that the integral of x— y over the upper half 
of the unit circle centred on the origin has the value 3. Show the same result 
by direct integration in Cartesian coordinates. 


11.5 Determine the point of intersection P, in the first quadrant, of the two ellipses 
ey ey 
Zz + po 1 and Bp + aa 1 


Taking b < a, consider the contour L that bounds the area in the first quadrant 
that is common to the two ellipses. Show that the parts of L that lie along the 
coordinate axes contribute nothing to the line integral around L of x dy — y dx. 
Using a parameterisation of each ellipse similar to that employed in the example 
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11.6 


11.8 


11.9 


11.10 


11.11 


in section 11.3, evaluate the two remaining line integrals and hence find the total 
area common to the two ellipses. 

By using parameterisations of the form x = acos" @ and y = asin" 0 for suitable 
values of n, find the area bounded by the curves 


32/5 $yPh= @ and x23 $3 = gl. 


Evaluate the line integral 
i ¢ [y(4x2 + y2) dx + x(2x? + 3y?) dy] 
c 


around the ellipse x?/a? + y?/b? = 1. 
Criticise the following ‘proof’ that z = 0. 


(a) Apply Green’s theorem in a plane to the functions P(x, y) = tan7!(y/x) and 
Q(x, y) = tan7!(x/y), taking the region R to be the unit circle centred on the 
origin. 

(b) The RHS of the equality so produced is 


y-x 
2o* axd 
ie vp 


which, either from symmetry considerations or by changing to plane polar 
coordinates, can be shown to have zero value. 

(c) In the LHS of the equality, set x = cos@ and y = sin@, yielding P(0) = 0 
and Q(0) = 2/2 — 0. The line integral becomes 


ie [($-9) cos 0 — 0 sin 0] dO, 
0 


which has the value 27. 
(d) Thus 2z = 0 and the stated result follows. 


A single-turn coil C of arbitrary shape is placed in a magnetic field B and carries 
a current I. Show that the couple acting upon the coil can be written as 


M=1 [(B-nde—1 | Beran, 


For a planar rectangular coil of sides 2a and 2b placed with its plane vertical 
and at an angle ¢ to a uniform horizontal field B, show that M is, as expected, 
4abBI cos dk. 

Find the vector area S of the part of the curved surface of the hyperboloid of 
revolution 


x2 y? + 7? 


eB 
that lies in the region z > 0 anda<x<da. 
An axially symmetric solid body with its axis AB vertical is immersed in an 
incompressible fluid of density po. Use the following method to show that, 
whatever the shape of the body, for = p(z) in cylindrical polars the Archimedean 
upthrust is, as expected, ppgV, where V is the volume of the body. 

Express the vertical component of the resultant force on the body, — { pd§, 
where p is the pressure, in terms of an integral; note that p = —pogz and that for 
an annular surface element of width dl, n-n, dl = —dp. Integrate by parts and 
use the fact that p(z4) = p(zg) = 0. 


=1 
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11.12 


11.13 


11.14 


11.15 


11.16 


11.17 


11.18 


Show that the expression below is equal to the solid angle subtended by a 
rectangular aperture, of sides 2a and 2b, at a point on the normal through its 
centre, and at a distance c from the aperture: 


ona[ Has dy 
"bo PHAGE +A? 


By setting y = (a? +c’)! tan, change this integral into the form 


*  daccos p 
dg, 
i b 


+a sin’ b 
where tan b; = b/(a? +c’)'/”, and hence show that 


| ab 
Q = 4tan - +P payin 
A vector field a is given by —zxri—zyrj+(x?+y?)r7k, where r? = x?+y?+27. 
Establish that the field is conservative (a) by showing that V x a = 0, and (b) by 
constructing its potential function ¢. 
A vector field a is given by (z7 + 2xy)i+ (x? + 2yz)j + (y? + 2zx)k. Show that 
a is conservative and that the line integral { a- dr along any line joining (1, 1,1) 
and (1,2,2) has the value 11. 
A force F(r) acts on a particle at r. In which of the following cases can F be 
represented in terms of a potential? Where it can, find the potential. 


ee a) 2 
(b) F= f [zk + > exp (-5) : 


() F=f ee 


One of Maxwell’s electromagnetic equations states that all magnetic fields B 
are solenoidal (ic. V- B = 0). Determine whether each of the following vectors 
could represent a real magnetic field; where it could, try to find a suitable vector 
potential A, ic. such that B = V x A. (Hint: seek a vector potential that is parallel 
to V x B.): 

Bob P - 2 2 : P wa 2 a eee 
(a) sa le yzit (x — y)zj + (x° — y~)k] in Cartesians with r° = x°+y°+2°; 

3 


Bob 
(b) = [cos 0 cos #é, — sin 0 cos P€y + sin20 sin $€g] in spherical polars; 


zp a 
Prep et 


The vector field f has components yi—xj+k and y is a curve given parametrically 
by 


(c) Bob? in cylindrical polars. 


r=(a—c+ccosO)it+ (b+csinO)j+cC0k, 0<0<2n. 


Describe the shape of the path y and show that the line integral [) f- dr vanishes. 
Does this result imply that f is a conservative field? 

A vector field a = f(r)r is spherically symmetric and everywhere directed away 
from the origin. Show that a is irrotational, but that it is also solenoidal only if 
f(r) is of the form Ar->. 
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11.19 


11.20 


11.21 


11.22 


11.23 


Evaluate the surface integral { r-dS, where r is the position vector, over that part 
of the surface z = a? — x? — y? for which z > 0, by each of the following methods. 


(a) Parameterise the surface as x = asin@ cos¢, y = asin@ sin, z = a’ cos” 0, 
and show that 


r- dS = a‘(2sin’ 0 cos 0 + cos* 0 sin 0) dO dd. 


(b) Apply the divergence theorem to the volume bounded by the surface and 
the plane z = 0. 


Obtain an expression for the value dp at a point P of a scalar function @ that 
satisfies V7 = 0, in terms of its value and normal derivative on a surface S that 
encloses it, by proceeding as follows. 


(a) In Green’s second theorem, take p at any particular point Q as 1/r, where r 
is the distance of Q from P. Show that Vp = 0, except at r = 0. 

(b) Apply the result to the doubly connected region bounded by S and a small 
sphere & of radius 6 centred on P. 

(c) Apply the divergence theorem to show that the surface integral over 2 
involving 1/6 vanishes, and prove that the term involving 1/57 has the value 
4nop. 

(d) Conclude that 


1 d/l 1 f 1a¢ 
or=-z [05 (5) Stel eee 


This important result shows that the value at a point P of a function ¢ 
that satisfies V>¢ = 0 everywhere within a closed surface S that encloses P 
may be expressed entirely in terms of its value and normal derivative on S. 
This matter is taken up more generally in connection with Green’s functions 
in chapter 21 and in connection with functions of a complex variable in 
section 24.10. 


Use result (11.21), together with an appropriately chosen scalar function ¢, to 
prove that the position vector r of the centre of mass of an arbitrarily shaped 
body of volume V and uniform density can be written 


Bees | 
r=7f 31° dS. 


A rigid body of volume V and surface S rotates with angular velocity w. Show that 


1 
So dS. 
o ay pe " 


where u(x) is the velocity of the point x on the surface S. 
Demonstrate the validity of the divergence theorem: 


(a) by calculating the flux of the vector 
or 


F= (Peta 


through the spherical surface |r| = /3a; 
(b) by showing that 


3a? 


(P+ a)? 
and evaluating the volume integral of V - F over the interior of the sphere 


|r| = 3a. The substitution r = atan6@ will prove useful in carrying out the 
integration. 


V-F= 
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11.24 


11.25 


11.26 


11.27 


11.28 


Prove equation (11.22) and, by taking b = zx7i+ zy?j + (x? — y”)k, show that the 
two integrals 


r= f 2av and J = | c0s*Osin’ dcos? 6 d0 dé. 


both taken over the unit sphere, must have the same value. Evaluate both directly 
to show that the common value is 47/15. 
In a uniform conducting medium with unit relative permittivity, charge density p, 
current density J, electric field E and magnetic field B, Maxwell’s electromagnetic 
equations take the form (with pigéo = c~”) 


(i) V-B=0, (ii) V-E=p/eo, 
(iii) Vx E+ B=0, (iv) Vx B—(E/c?) = pod. 


The density of stored energy in the medium is given by $(e9E? + y)'B?). Show 
that the rate of change of the total stored energy in a volume V is equal to 


- [ seav—— fExB) ds, 
V Lo Js 


where S is the surface bounding V. 

[The first integral gives the ohmic heating loss, whilst the second gives the 
electromagnetic energy flux out of the bounding surface. The vector yj '(E x B) 
is known as the Poynting vector. ] 


A vector field F is defined in cylindrical polar coordinates p,0,z by 


F=f (Sei a ec iz)k) = 10? (cos Az)e, + Fo(sin Az)k, 
a a a 


where i, j and k are the unit vectors along the Cartesian axes and e, is the unit 
vector (x/p)i + (y/p)j. 


(a) Calculate, as a surface integral, the flux of F through the closed surface 
bounded by the cylinders p = a and p = 2a and the planes z = tan/2. 
(b) Evaluate the same integral using the divergence theorem. 


The vector field F is given by 
F = (3x7yz + y?z + xe i + (3xy2z +32 + ye’)j + (x3 y + y3x + xy?27)k. 


Calculate (a) directly, and (b) by using Stokes’ theorem the value of the line 
integral {, F - dr, where L is the (three-dimensional) closed contour OABCDEO 
defined by the successive vertices (0,0, 0), (1,0,0), (1,0, 1), (1, 1, 1), (1, 1,0), (0, 1, 0), 
(0,0, 0). 

A vector force field F is defined in Cartesian coordinates by 


3 5 2 ‘ 7 
F=f ( YY oole | i) i4 (3 i SEP nie) i Z oxv/a K 
a a a a 
Use Stokes’ theorem to calculate 
fr - dr, 
L 


where L is the perimeter of the rectangle ABCD given by A = (0, 1,0), B = (1, 1,0), 
C = (1,3,0) and D = (0,3,0). 
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11.11 Hints and answers 


Show that V x F = 0. The potential ¢p(r) = x?z + y?z? —z. 

(a) 8 In2i + 2j + (3c/2)k; (b) (—3c*/8)i — cj — (c? In 2)k; (c) c+ In2—c. 

For P, x = y = ab/(a? + b’)'/?. The relevant limits are 0 < 0, < tan7!(b/a) and 
tan“!(a/b) < 02 < 2/2. The total common area is 4ab tan”'(b/a). 

Show that, in the notation of section 11.3, 6Q/éx — oP /dy = = 2x*; I = na*b/2. 
M=I for x (dr x B). Show that the horizontal sides in the first term and the 
whole of the second term contribute nothing to the couple. 

Note that, if i is the outward normal to the surface, fi, - iid! is equal to —dp. 
(b) $= +2/r. 

(a) Yes, Fo(x — y)exp(—r?/a’); (b) yes, —Fol(x? + y*)/(2a)] exp(—r?/a’); 

(c)no, Vx F#0. 

A spiral of radius c with its axis parallel to the z-direction and passing through 
(a,b). The pitch of the spiral is 2xc?. No, because (i) y is not a closed loop and 
(ii) the line integral must be zero for every closed jaar: not just for a particular 
one. In fact V x f = —2k # 0 shows that f is not conservative. 

(a) dS = (2a cos 0 sin? 0 cos pi + 2a’ cos @ sin? 6 sin fj + a? cos 0 sin 0k) dO dd. 

(b) V-r = 3; over the plane z = 0, r-dS = 0. 

The necessarily common value is 3za*/2. 

Write r as V(4r7). 

The answer is 3,/32a/2 in each case. 

Identify the expression for V - (E x B) and use the divergence theorem. 

(a) The successive contributions to the integral are: 

1—2e-4, eae *, —1+42e1, —5. 
(b) V x F = 2xyz7i— y’z?j + ye*k. Show that the contour is equivalent to the 
sum of two plane square contours in the planes z = 0 and x = 1, the latter being 
traversed in the negative sense. Integral = ze —5). 
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Fourier series 


We have already discussed, in chapter 4, how complicated functions may be 
expressed as power series. However, this is not the only way in which a function 
may be represented as a series, and the subject of this chapter is the expression 
of functions as a sum of sine and cosine terms. Such a representation is called a 
Fourier series. Unlike Taylor series, a Fourier series can describe functions that are 
not everywhere continuous and/or differentiable. There are also other advantages 
in using trigonometric terms. They are easy to differentiate and integrate, their 
moduli are easily taken and each term contains only one characteristic frequency. 
This last point is important because, as we shall see later, Fourier series are often 
used to represent the response of a system to a periodic input, and this response 
often depends directly on the frequency content of the input. Fourier series are 
used in a wide variety of such physical situations, including the vibrations of a 
finite string, the scattering of light by a diffraction grating and the transmission 
of an input signal by an electronic circuit. 


12.1 The Dirichlet conditions 


We have already mentioned that Fourier series may be used to represent some 
functions for which a Taylor series expansion is not possible. The particular 
conditions that a function f(x) must fulfil in order that it may be expanded as a 
Fourier series are known as the Dirichlet conditions, and may be summarised by 
the following four points: 


(i) the function must be periodic; 
(ii) it must be single-valued and continuous, except possibly at a finite number 
of finite discontinuities; 
(iii) it must have only a finite number of maxima and minima within one 
period; 
(iv) the integral over one period of |f(x)| must converge. 
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Figure 12.1 An example of a function that may be represented as a Fourier 
series without modification. 


If the above conditions are satisfied then the Fourier series converges to f(x) 
at all points where f(x) is continuous. The convergence of the Fourier series 
at points of discontinuity is discussed in section 12.4. The last three Dirichlet 
conditions are almost always met in real applications, but not all functions are 
periodic and hence do not fulfil the first condition. It may be possible, however, 
to represent a non-periodic function as a Fourier series by manipulation of the 
function into a periodic form. This is discussed in section 12.5. An example of 
a function that may, without modification, be represented as a Fourier series is 
shown in figure 12.1. 

We have stated without proof that any function that satisfies the Dirichlet 
conditions may be represented as a Fourier series. Let us now show why this is 
a plausible statement. We require that any reasonable function (one that satisfies 
the Dirichlet conditions) can be expressed as a linear sum of sine and cosine 
terms. We first note that we cannot use just a sum of sine terms since sine, being 
an odd function (i.e. a function for which f(—x) = —f(x)), cannot represent even 
functions (i.e. functions for which f(—x) = f(x)). This is obvious when we try 
to express a function f(x) that takes a non-zero value at x = 0. Clearly, since 
sinnx = 0 for all values of n, we cannot represent f(x) at x = 0 by a sine series. 
Similarly odd functions cannot be represented by a cosine series since cosine is 
an even function. Nevertheless, it is possible to represent all odd functions by a 


sine series and all even functions by a cosine series. Now, since all functions may 
be written as the sum of an odd and an even part, 


F(x) 


ll 


S1f (0) + f(—)] + $1f 0) — fH) 
feven(x) te foaa(*), 


ll 
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we can write any function as the sum of a sine series and a cosine series. 
All the terms of a Fourier series are mutually orthogonal, i.e. the integrals, over 
one period, of the product of any two terms have the following properties: 


xXo+L 2 2 
/ sin ( =) cos (=) dx =0 for all r and p, (12.1) 
Hs L L 


L forr=p=0 
XotL > 
/ cos (=) cos (==) dx = 5L forr=p>0, (12.2) 
ca 0 for r # p, 
0 forr=p=0 
Xo +L 2 2 > 
i sin ( J sin ( oe) dx = 5L forr =p>0, (12.3) 
mo 0 forr#p, 


where r and p are integers greater than or equal to zero; these formulae are easily 
derived. A full discussion of why it is possible to expand a function as a sum of 
mutually orthogonal functions is given in chapter 17. 

The Fourier series expansion of the function f(x) is conventionally written 


4 2 2nrx ; 2nrx 
to E fm) ome CE) om 


where do, a,,b, are constants called the Fourier coefficients. These coefficients are 
analogous to those in a power series expansion and the determination of their 


numerical values is the essential step in writing a function as a Fourier series. 

This chapter continues with a discussion of how to find the Fourier coefficients 
for particular functions. We then discuss simplifications to the general Fourier 
series that may save considerable effort in calculations. This is followed by the 
alternative representation of a function as a complex Fourier series, and we 
conclude with a discussion of Parseval’s theorem. 


12.2 The Fourier coefficients 


We have indicated that a series that satisfies the Dirichlet conditions may be 
written in the form (12.4). We now consider how to find the Fourier coefficients 
for any particular function. For a periodic function f(x) of period L we will find 
that the Fourier coefficients are given by 


Xo +L 

p= / f(x) cos (=) de (12.5) 
xo 
Xo +L 

a - i, Aan (=) is (12.6) 
xo 


where Xo is arbitrary but is often taken as 0 or —L/2. The apparently arbitrary 


factor 5 which appears in the ag term in (12.4) is included so that (12.5) may 
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apply for r = 0 as well as r > 0. The relations (12.5) and (12.6) may be derived 
as follows. 
Suppose the Fourier series expansion of f(x) can be written as in (12.4), 


f(x) = + <2 c cos (=) + b, sin (=)| : 


Then, multiplying by cos(2zpx/L), integrating over one full period in x and 
changing the order of the summation and integration, we get 


xXot+L 2) Xot+L 2 
/ f(x) cos (=) dx = 2 / cos ( -) dx 
x0 L 2 x0 L 

= xoth 2 2 

+a f cos ( T) cos ( -) dx 
r=1 *0 
00 Xo+L 2 

+>) b, f sin ( = 
r=1 x0 


(12.7) 


We can now find the Fourier coefficients by considering (12.7) as p takes different 
values. Using the orthogonality conditions (12.1)-(12.3) of the previous section, 
we find that when p = 0 (12.7) becomes 


Xo +L 
/ f (x)dx = SL. 


0 


When p # 0 the only non-vanishing term on the RHS of (12.7) occurs when 


r = p, and so 
ane 2 
[ f (x) cos (=) dx = SL. 


The other Fourier coefficients b, may be found by repeating the above process 
but multiplying by sin(2zpx/L) instead of cos(27px/L) (see exercise 12.2). 


> Express the square-wave function illustrated in figure 12.2 as a Fourier series. 


Physically this might represent the input to an electrical circuit that switches between a 
high and a low state with time period T. The square wave may be represented by 


—1 for -}T <t <0, 
no= 44 forO<¢<iT. 


In deriving the Fourier coefficients, we note firstly that the function is an odd function 
and so the series will contain only sine terms (this simplification is discussed further in the 
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f() 


1 


| 
NII 
S| 
v5 

+ 


Figure 12.2 A square-wave function. 


following section). To evaluate the coefficients in the sine series we use (12.6). Hence 


T/2 
heey Feain (=) dt 
ra ee T 


ff sin 2nrt\ 
Fy T 
2 


= L—(-1) 1: 


Thus the sine coefficients are zero if r is even and equal to 4/(zr) if r is odd. Hence the 
Fourier series for the square-wave function may be written as 


(12.8) 


4/. sin3mt  sinS5at 
f() =—-(sinot+ + fee), 
T 


3 5 


where w = 27/T is called the angular frequency. < 


12.3 Symmetry considerations 


The example in the previous section employed the useful property that since the 
function to be represented was odd, all the cosine terms of the Fourier series were 
absent. It is often the case that the function we wish to express as a Fourier series 
has a particular symmetry, which we can exploit to reduce the calculational labour 
of evaluating Fourier coefficients. Functions that are symmetric or antisymmetric 
about the origin (i.e. even and odd functions respectively) admit particularly 
useful simplifications. Functions that are odd in x have no cosine terms (see 
section 12.1) and all the a-coefficients are equal to zero. Similarly, functions that 
are even in x have no sine terms and all the b-coefficients are zero. Since the 
Fourier series of odd or even functions contain only half the coefficients required 
for a general periodic function, there is a considerable reduction in the algebra 
needed to find a Fourier series. 

The consequences of symmetry or antisymmetry of the function about the 
quarter period (i.e. about L/4) are a little less obvious. Furthermore, the results 
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are not used as often as those above and the remainder of this section can be 
omitted on a first reading without loss of continuity. The following argument 
gives the required results. 

Suppose that f(x) has even or odd symmetry about L/4, ie. f(L/4— x) = 
+f(x — L/4). For convenience, we make the substitution s = x — L/4 and hence 
f(—s) = +f(s). We can now see that 


2 pork Ons oar 
b=z f Foo)sin( L +3) ds, 


0 


where the limits of integration have been left unaltered since f is, of course, 
periodic in s as well as in x. If we use the expansion 


ae 2mrs _ ar ae 2nrs cos (3) + cos 2mrs sin (™") 
i , L 2 L 2)’ 


we can immediately see that the trigonometric part of the integrand is an odd 
function of s if r is even and an even function of s if r is odd. Hence if f(s) is 
even and r is even then the integral is zero, and if f(s) is odd and r is odd then 
the integral is zero. Similar results can be derived for the Fourier a-coefficients 
and we conclude that 


(i) if f(x) is even about L/4 then az,,; = 0 and bo, = 0, 
(ii) if f(x) is odd about L/4 then ay, = 0 and b2,4; = 0. 


All the above results follow automatically when the Fourier coefficients are 
evaluated in any particular case, but prior knowledge of them will often enable 
some coefficients to be set equal to zero on inspection and so substantially reduce 
the computational labour. As an example, the square-wave function shown in 
figure 12.2 is (1) an odd function of t, so that all a, = 0, and (ii) even about the 
point t = T/4, so that bj, = 0. Thus we can say immediately that only sine terms 
of odd harmonics will be present and therefore will need to be calculated; this is 
confirmed in the expansion (12.8). 


12.4 Discontinuous functions 


The Fourier series expansion usually works well for functions that are discon- 
tinuous in the required range. However, the series itself does not produce a 
discontinuous function and we state without proof that the value of the ex- 
panded f(x) at a discontinuity will be half-way between the upper and lower 
values. Expressing this more mathematically, at a point of finite discontinuity, xq, 
the Fourier series converges to 


dlim[f(xu +6) + feu — 6) 


At a discontinuity, the Fourier series representation of the function will overshoot 
its value. Although as more terms are included the overshoot moves in position 
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Figure 12.3 The convergence of a Fourier series expansion of a square-wave 
function, including (a) one term, (b) two terms, (c) three terms and (d) 20 
terms. The overshoot 6 is shown in (d). 


arbitrarily close to the discontinuity, it never disappears even in the limit of an 
infinite number of terms. This behaviour is known as Gibbs’ phenomenon. A full 
discussion is not pursued here but suffice it to say that the size of the overshoot 
is proportional to the magnitude of the discontinuity. 


> Find the value to which the Fourier series of the square-wave function discussed in sec- 
tion 12.2 converges at t = 0. 


It can be seen that the function is discontinuous at t = 0 and, by the above rule, we expect 
the series to converge to a value half-way between the upper and lower values, in other 
words to converge to zero in this case. Considering the Fourier series of this function, 
(12.8), we see that all the terms are zero and hence the Fourier series converges to zero as 
expected. The Gibbs phenomenon for the square-wave function is shown in figure 12.3. < 
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Figure 12.4 Possible periodic extensions of a function. 


12.5 Non-periodic functions 


We have already mentioned that a Fourier representation may sometimes be used 
for non-periodic functions. If we wish to find the Fourier series of a non-periodic 
function only within a fixed range then we may continue the function outside the 
range so as to make it periodic. The Fourier series of this periodic function would 
then correctly represent the non-periodic function in the desired range. Since we 
are often at liberty to extend the function in a number of ways, we can sometimes 
make it odd or even and so reduce the calculation required. Figure 12.4(b) shows 
the simplest extension to the function shown in figure 12.4(a). However, this 
extension has no particular symmetry. Figures 12.4(c), (d) show extensions as odd 
and even functions respectively with the benefit that only sine or cosine terms 
appear in the resulting Fourier series. We note that these last two extensions give 
a function of period 2L. 

In view of the result of section 12.4, it must be added that the continuation 
must not be discontinuous at the end-points of the interval of interest; if it is 
the series will not converge to the required value there. This requirement that 
the series converges appropriately may reduce the choice of continuations. This 
is discussed further at the end of the following example. 


> Find the Fourier series of f(x) =x? for0<x <2. 


We must first make the function periodic. We do this by extending the range of interest to 
—2 <x <2 in such a way that f(x) = f(—x) and then letting f(x + 4k) = f(x), where k is 
any integer. This is shown in figure 12.5. Now we have an even function of period 4. The 
Fourier series will faithfully represent f(x) in the range, —2 < x < 2, although not outside 
it. Firstly we note that since we have made the specified function even in x by extending 
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f(x) =x? 


L 


Figure 12.5 f(x) =x?,0 <x < 2, with the range extended to give periodicity. 


the range, all the coefficients b, will be zero. Now we apply (12.5) and (12.6) with L = 4 
to determine the remaining coefficients: 


2 f?-5 2arx 4 ft mx 
a= 5 fx cos ( 7 ) a= Ff 2008 (5) a 


where the second equality holds because the function is even in x. Thus 
2 5... (mIrx 4 [? . ¢arx 
a Es sin ( 2 J =[ xsin (*) as 
8 mrx\ 12 8 : Trx 
mr? [x<os ( 2 Yo ae cos (>) oe 


2 


Since this expression for a, has r? in its denominator, to evaluate a) we must return to the 


original definition, 
a PP? 
a, = ri [ fesveos (=) dx. 


From this we obtain 


The final expression for f(x) is then 


4 = (-1)" TIX 
2 
x a ree 7 cos (=) for0<x<2.< 


We note that in the above example we could have extended the range so as 
to make the function odd. In other words we could have set f(x) = —f(—x) and 
then made f(x) periodic in such a way that f(x +4) = f(x). In this case the 
resulting Fourier series would be a series of just sine terms. However, although 
this will faithfully represent the function inside the required range, it does not 
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converge to the correct values of f(x) = +4 at x = +2; it converges, instead, to 
zero, the average of the values at the two ends of the range. 


12.6 Integration and differentiation 


It is sometimes possible to find the Fourier series of a function by integration or 
differentiation of another Fourier series. If the Fourier series of f(x) is integrated 
term by term then the resulting Fourier series converges to the integral of f(x). 
Clearly, when integrating in such a way there is a constant of integration that must 
be found. If f(x) is a continuous function of x for all x and f(x) is also periodic 
then the Fourier series that results from differentiating term by term converges to 
f'(x), provided that f’(x) itself satisfies the Dirichlet conditions. These properties 
of Fourier series may be useful in calculating complicated Fourier series, since 
simple Fourier series may easily be evaluated (or found from standard tables) 
and often the more complicated series can then be built up by integration and/or 
differentiation. 


> Find the Fourier series of f(x) =x° for0<x <2. 


In the example discussed in the previous section we found the Fourier series for f(x) = x? 
in the required range. So, if we integrate this term by term, we obtain 


x = ax4+32). CY sin (=) He 
r=1 


where c is, so far, an arbitrary constant. We have not yet found the Fourier series for x* 
because the term tx appears in the expansion. However, by now differentiating the same 
initial expression for x? we obtain 


“(-1)" sp arx 
2x = —8 —). 
3 dX rr a ( 2 ) 
We can now write the full Fourier expansion of x? as 
3 “(1 parx “(-1) yp arx 
x= ape Sao (> ) 0 De 3,3 sin (> ) +e. 


Finally, we can find the constant, c, by considering f(0). At x = 0, our Fourier expansion 
gives x* = since all the sine terms are zero, and hence c = 0. < 


12.7 Complex Fourier series 


As a Fourier series expansion in general contains both sine and cosine parts, it 
may be written more compactly using a complex exponential expansion. This 
simplification makes use of the property that exp(irx) = cosrx + isinrx. The 
complex Fourier series expansion is written 

foe) 


fo) = So crexp (=). (12.9) 


r=—oo 
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where the Fourier coefficients are given by 


1 ror 2nirx 
= 7 a f(x) exp (- L ) dx. (12.10) 


This relation can be derived, in a similar manner to that of section 12.2, by mul- 
tiplying (12.9) by exp(—2zipx/L) before integrating and using the orthogonality 


relation 
ee ( tae (=) F L forr=p, 
exp exp = 
Xo L L 0 forrF#p. 


The complex Fourier coefficients in (12.9) have the following relations to the real 
Fourier coefficients: 


c, = 4(a, — ib,), 
; (12.11) 
c_, = 5(a, + ib;). 


Note that if f(x) is real then c_, = cy, 
conjugation. 


> Find a complex Fourier series for f(x) =x in the range —2 < x <2. 


Using (12.10), for r # 0, 


= cf. Tirx d 
Cr = Z ner 5) x 
x 2 Tirx ths a Tirx ae 
2nir 7 2nir r 2 


1 ; : 1 nirx \ |? 
seared [exp(—zir) + exp(zir)] + Fes exp (-)] y 


where the asterisk represents complex 


= — cosar— sek sin zr = 4 ay. (12.12) 
hay r mr 


We note that the Fourier series derived for x in section 12.6 gives a, = 0 for all r and 


4-1 
mr 


— 


and so, using (12.11), we confirm that c, and c_, have the forms derived above. It is also 
apparent that the relationship c; = c_, holds, as we expect since f(x) is real. < 
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12.8 Parseval’s theorem 


Parseval’s theorem gives a useful way of relating the Fourier coefficients to the 
function that they describe. Essentially a conservation law, it states that 


al  pootax = So lel? 


x0 r=—0o 


= (day)? +4 (a2 +02). (12.13) 
r=1 


In a more memorable form, this says that the sum of the moduli squared of 
the complex Fourier coefficients is equal to the average value of |f(x)|? over one 
period. Parseval’s theorem can be proved straightforwardly by writing f(x) as 
a Fourier series and evaluating the required integral, but the algebra is messy. 
Therefore, we shall use an alternative method, for which the algebra is simple 
and which in fact leads to a more general form of the theorem. 

Let us consider two functions f(x) and g(x), which are (or can be made) 
periodic with period L and which have Fourier series (expressed in complex 
form) 


a 


fox) = D> exp (“), 


r=—% 
ae 3 ee (=) 
& con r EXP L > 


where c, and y, are the complex Fourier coefficients of f(x) and g(x) respectively. 
Thus 


ea) 


9 
feos = Yo ce"(sdexp ( wy). 


r=—oo 


Integrating this equation with respect to x over the interval (xo,xo + L) and 
dividing by L, we find 


ll 


< ae *(x) ex 2nirx ix 
Or x i 
CE. er aE 


r=—0o 


2 c a ie (x) ex Sra dx , 
rae g p L 


r=—00 xo 


Xot+L 
tf feve'ooax 


ll 


ll 
ag 
2 


where the last equality uses (12.10). Finally, if we let g(x) = f(x) then we obtain 
Parseval’s theorem (12.13). This result can be proved in a similar manner using 
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the sine and cosine form of the Fourier series, but the algebra is slightly more 
complicated. 

Parseval’s theorem is sometimes used to sum series. However, if one is presented 
with a series to sum, it is not usually possible to decide which Fourier series 
should be used to evaluate it. Rather, useful summations are nearly always found 
serendipitously. The following example shows the evaluation of a sum by a 
Fourier series method. 


> Using Parseval’s theorem and the Fourier series for f(x) = x? found in section 12.5, 
calculate the sum \>*°_,r~*+ 


Firstly we find the average value of [f(x)]? over the interval —2 < x <2: 


Now we evaluate the right-hand side of (12.13): 
(Jay? +E e+ EY w= (8) +40 See 
1 1 r=1 


Equating the two expression we find 


a 


oe 


12.9 Exercises 


12.1 Prove the orthogonality relations stated in section 12.1. 

{2,2 Derive the Fourier coefficients b, in a similar manner to the derivation of the a, 
in section 12.2. 

12.3 Which of the following functions of x could be represented by a Fourier series 


over the range indicated? 


(a) tanh7!(x), —-0 <x <0; 
(b) tan x, —-0 <x <0; 
(c) | sin x|-'/?, —0 <x <0; 
(d) cos!(sin 2x), —0<x<0; 
(e) xsin(1/x), —n1<x <1, cyclically repeated. 
12.4 By moving the origin of t to the centre of an interval in which f(t) = +1, ie. 


by changing to a new independent variable t/ = t— iT, express the square-wave 
function in the example in section 12.2 as a cosine series. Calculate the Fourier 
coefficients involved (a) directly and (b) by changing the variable in result (12.8). 

12.5 Find the Fourier series of the function f(x) = x in the range —z < x < z. Hence 
show that 


ee ee en 
BF 56 7 4 

12.6 For the function 
f(x)=1-~x, 0<x<l, 


find (a) the Fourier sine series and (b) the Fourier cosine series. Which would 
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12.7 


12.8 


12.9 


12.10 


12.11 


12,12. 


12.13 


12.14 


be better for numerical evaluation? Relate your answer to the relevant periodic 
continuations. 
For the continued functions used in exercise 12.6 and the derived corresponding 
series, consider (i) their derivatives and (ii) their integrals. Do they give meaningful 
equations? You will probably find it helpful to sketch all the functions involved. 
The function y(x) = xsinx for 0 < x < z is to be represented by a Fourier series 
of period 27 that is either even or odd. By sketching the function and considering 
its derivative, determine which series will have the more rapid convergence. Find 
the full expression for the better of these two series, showing that the convergence 
~ n> and that alternate terms are missing. 
Find the Fourier coefficients in the expansion of f(x) = exp x over the range 
—1 <x <1. What value will the expansion have when x = 2? 
By integrating term by term the Fourier series found in the previous question 
and using the Fourier series for f(x) = x found in section 12.6, show that 
fexpxdx = expx +c. Why is it not possible to show that d(expx)/dx = expx 
by differentiating the Fourier series of f(x) = expx in a similar manner? 
Consider the function f(x) = exp(—x?) in the range 0 < x < 1. Show how it 
should be continued to give as its Fourier series a series (the actual form is not 
wanted) (a) with only cosine terms, (b) with only sine terms, (c) with period 1 
and (d) with period 2. 

Would there be any difference between the values of the last two series at (i) 
x =0, (ii) x = 1? 
Find, without calculation, which terms will be present in the Fourier series for 
the periodic functions f(t), of period T, that are given in the range —T/2 to T/2 
by: 
(a) f(t)=2 for 0< |t| < T/4, f =1 for T/4 < |t| < T/2; 
(b) f(t) = expl-(t— 7/4); 
(c) f(t) = —1 for —T/2 < t < —3T/8 and 3T/8 < t < T/2,f(t) = 1 for 

—T/8 <t < T/8; the graph of f is completed by two straight lines in the 
remaining ranges so as to form a continuous function. 


Consider the representation as a Fourier series of the displacement of a string 
lying in the interval 0 < x < L and fixed at its ends, when it is pulled aside by yo 
at the point x = L/4. Sketch the continuations for the region outside the interval 
that will 


(a) produce a series of period L, 
(b) produce a series that is antisymmetric about x = 0, and 
(c) produce a series that will contain only cosine terms. 
(d) What are (i) the periods of the series in (b) and (c) and (ii) the value of the 
“ag-term’ in (c)? 

(e) Show that a typical term of the series obtained in (b) is 

32yo nm . mmx 

sin — sin —. 

3n2n? 4 L 
Show that the Fourier series for the function y(x) = |x| in the range -—1 <x<az 
is 


_nm 4 “ cos(2m + 1)x 
y(x) 5 ae Om+ IP 


By integrating this equation term by term from 0 to x, find the function g(x) 
whose Fourier series is 


ee sin(2m + 1)x 
“Qm+ 13” 
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12.15 


12.16 


12.17 


12.18 


12.19 


12.20 


Deduce the value of the sum S of the series 
1 1 1 
a 3 a 3 r BR | eee 


Using the result of exercise 12.14, determine, as far as possible by inspection, the 
forms of the functions of which the following are the Fourier series: 


(a) 


1 


1 1 
cos 6 + 5 cos30 + 5 cos5é +++: : 


(b) 


‘ 1. 1. 
sin 0 + 37 sin30 + To5 S150 + : 
(c) 
LV 41? mx 1 2nx 1 32x 
crthiy hee es © 00 oi 
(You may find it helpful to first set x = 0 in the quoted result and so obtain 
values for S, = )>(2m+1)~? and other sums derivable from it.) 


By finding a cosine Fourier series of period 2 for the function f(t) that takes the 
form f(t) = cosh(t — 1) in the range 0 < t < 1, prove that 


— wr? +1 ~ ea] 
Deduce values for the sums )7(n’x? + 1)~! over odd n and even n separately. 
Find the (real) Fourier series of period 2 for f(x) = coshx and g(x) = x? in the 
range —1 < x < 1. By integrating the series for f(x) twice, prove that 


3 (-1)""! 7 1 1 5 
mn(nen2+1) 2\sinhl 6/)° 


n=1 


Express the function f(x) = x? as a Fourier sine series in the range 0 < x < 2 
and show that it converges to zero at x = +2. 

Demonstrate explicitly for the square-wave function discussed in section 12.2 that 
Parseval’s theorem (12.13) is valid. You will need to use the relationship 


Show that a filter that transmits frequencies only up to 82/T will still transmit 
more than 90% of the power in such a square-wave voltage signal. 
Show that the Fourier series for | sin 6| in the range —z < 0 < z is given by 


4 & cos2m0 


T 4m? — 1° 
m=1 


| sin 0] 


By setting 0 = 0 and 0 = 1/2, deduce values for 


Dae ane » 16m — 1° 


TR] 


429 


FOURIER SERIES 


12.21 


12.22 


12.23 


12.24 


12.25 


12.26 


Find the complex Fourier series for the periodic function of period 27 defined in 
the range —n <x <7 by y(x) = coshx. By setting x = 0 prove that 


< (—1)" 
Siac 3 (aes 1). 


n=1 


The repeating output from an electronic oscillator takes the form of a sine wave 
f(t) = sint for 0 < t < 2/2; it then drops instantaneously to zero and starts 
again. The output is to be represented by a complex Fourier series of the form 


a) 
: Cn efnti, 


n=—o0 


Sketch the function and find an expression for c,. Verify that c_, = ci. Demon- 
strate that setting t = 0 and t = 2/2 produces differing values for the sum 


[ee) 1 
> 16n2 —1° 


Determine the correct value and check it using the result of exercise 12.20. 
Apply Parseval’s theorem to the series found in the previous exercise and so 
derive a value for the sum of the series 


17 65 145 16n? +1 


(5? ' (32° 043? °°" G6e—1P 
A string, anchored at x = +L/2, has a fundamental vibration frequency of 2L/c, 
where c is the speed of transverse waves on the string. It is pulled aside at its 
centre point by a distance yo and released at time t = 0. Its subsequent motion 
can be described by the series 


x 
nx nict 
V(x, t) = » An COS TT; cos Ee 


n=1 


Find a general expression for a, and show that only the odd harmonics of the 
fundamental frequency are present in the sound generated by the released string. 
By applying Parseval’s theorem, find the sum S of the series )0¢ (2m + 1)~*. 
Show that Parseval’s theorem for two real functions whose Fourier expansions 
have cosine and sine coefficients a,, b, and a, B, takes the form 


L oe) 
= [/ fo98" 6) de = arm + 5 ttn + ba) 


n=1 


(a) Demonstrate that for g(x) = sinmx or cosmx this reduces to the definition 
of the Fourier coefficients. 

(b) Explicitly verify the above result for the case in which f(x) = x and g(x) is 
the square-wave function, both in the interval -1 <x <1. 


An odd function f(x) of period 2z is to be approximated by a Fourier sine series 
having only m terms. The error in this approximation is measured by the square 
deviation 


n=1 


m 2 
Ey = / J — ye b, sin nl dx. 


By differentiating E,, with respect to the coefficients b,, find the values of b, that 
minimise E,,. 
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(a) (b) (c) (d) 


Figure 12.6 Continuations of exp(—x’) in 0 < x < 1 to give: (a) cosine terms 
only; (b) sine terms only; (c) period 1; (d) period 2. 


Sketch the graph of the function f(x), where 


_ f-x(n+x) for -t<x<0, 
fo) = { ES forO<x<xz. 


If f(x) is to be approximated by the first three terms of a Fourier sine series, what 
values should the coefficients have so as to minimise E;? What is the resulting 
value of E3? 


12.10 Hints and answers 


12.1 Note that the only integral of a sinusoid around a complete cycle of length L 
that is not zero is the integral of cos(2anx/L) when n = 0. 

12.3 Only (c). In terms of the Dirichlet conditions (section 12.1), the others fail as 
follows: (a) (i); (b) (ii); (d) (ii); (e) (iii). 

12.5 f(x) = 207 (-1)""'n“! sinnx; set x = 2/2. 

12.7 (i) Series (a) from exercise 12.6 does not converge and cannot represent the 
function y(x) = —1. Series (b) reproduces the square-wave function of equation 

12.8). 

ia eg (a) gives the series for y(x) = —x — 3x? - ; in the range -1<x<0 
and for y(x)=x— 5x? _ 5 in the range 0 < x < 1. Series (b) gives the series for 
y(x) =x + 4x? + 4 in the range —1 < x <0 and for y(x) = x — $x? + } in the 
rangeO<x<1. 

12.9 f(x) = (sinh 1) {1+ 2>07(—1)"(1 + nx?) [cos(nzx) — nx sin(nnx)] }. 

The series will converge to the same value as it does at x = 0, ie. f(0) = 1. 

12.11 See figure 12.6. (c) (i) (1 +e7!)/2, (ii) (1 + e7!)/2; (d) @ (1 + e~*)/2, (ii) eT. 

12.13 (d) (i) The periods are both 2L; (ii) yo/2. 

12.15  S, = 27/8. If S, = 3>(2m)~* then S, = 7 (Se. + S,), yielding S, — S. = 27/12 and 
S. +S. = 17/6. 

(a) (17/4)(1/2—|0); (b) (20/4)(x/2—|0|/2) from integrating (a). (c) Even function; 
average value L7/3; (0) = 0;y(L) = L’; probably y(x) = x?. Compare with the 
worked example in section 12.5. 

12.17 coshx = (sinh 1)[1+2 >” ,(—1)"(cosnzx)/(n’n? + 1)] and after integrating twice 
this form must be recovered. Use x? = $+4)>(—1)"(cos nzx)/(n?n7)] to eliminate 
the quadratic term arising from the constants of integration; there is no linear 
term. 

12.19 Ceeme1) = F2i/[(2m + 1)z]; 35 |C,? = (4/2?) x 2 x (n?/8); the values n = +1, 
+3 contribute > 90% of the total. 
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12.21 Cn = [(—1)" sinh z]/[z(1 + n’)]. Having set x = 0, separate out the n = 0 term 
and note that (—1)" =(—1)™. 

12.23. (x? —8)/16. 

12.25 (b) All a, and «, are zero; b, = 2(—1)"*!/(nn) and B, = 4/(nz). You will need 
the result quoted in exercise 12.19. 
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Integral transforms 


In the previous chapter we encountered the Fourier series representation of a 
periodic function in a fixed interval as a superposition of sinusoidal functions. It is 
often desirable, however, to obtain such a representation even for functions defined 
over an infinite interval and with no particular periodicity. Such a representation 
is called a Fourier transform and is one of a class of representations called integral 
transforms. 

We begin by considering Fourier transforms as a generalisation of Fourier 
series. We then go on to discuss the properties of the Fourier transform and its 
applications. In the second part of the chapter we present an analogous discussion 
of the closely related Laplace transform. 


13.1 Fourier transforms 


The Fourier transform provides a representation of functions defined over an 
infinite interval and having no particular periodicity, in terms of a superposition 
of sinusoidal functions. It may thus be considered as a generalisation of the 
Fourier series representation of periodic functions. Since Fourier transforms are 
often used to represent time-varying functions, we shall present much of our 
discussion in terms of f(t), rather than f(x), although in some spatial examples 
f(x) will be the more natural notation and we shall use it as appropriate. Our 
only requirement on f(t) will be that ies |f()| dt is finite. 

In order to develop the transition from Fourier series to Fourier transforms, we 
first recall that a function of period T may be represented as a complex Fourier 
series, cf. (12.9), 


ee) 


{OS > Ger = 3 ern (13.1) 


r=—o0 r=—oo0 


where w, = 2ar/T. As the period T tends to infinity, the ‘frequency quantum’ 
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c(@) exp iwt 


Figure 13.1 The relationship between the Fourier terms for a function of 
period T and the Fourier integral (the area below the solid line) of the 
function. 


Aw = 2x/T becomes vanishingly small and the spectrum of allowed frequencies 
jw, becomes a continuum. Thus, the infinite sum of terms in the Fourier series 
becomes an integral, and the coefficients c, become functions of the continuous 
variable w, as follows. 

We recall, cf. (12.10), that the coefficients c, in (13.1) are given by 


A T/2 : 
== +f f(the2nl dt = SS fie" dt, (13.2) 
Qn Jr 
where we have written the integral in two alternative forms and, for convenience, 
made one period run from —T/2 to +T/2 rather than from 0 to T. Substituting 
from (13.2) into (13.1) gives 


f(t)= p2 Ae ) f(u) et" du el", (13.3) 


At this stage «, is still a discrete function of r equal to 2xr/T. 

The solid points in figure 13.1 are a plot of (say, the real part of) c, e 
a function of r (or equivalently of w,) and it is clear that (22/T)c, e!"' gives 
the area of the rth broken-line rectangle. If T tends to oo then Aw (= 22/T) 
becomes infinitesimal, the width of the rectangles tends to zero and, from the 
mathematical definition of an integral, 


' as 


ice) ‘00 


Aw ia, t 1 iat 
De a elone > xf g(w)e de. 


r=—o oO: 


In this particular case 
T/2 ; 
2(@,) = ih fie" du, 
—T/2 
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and (13.3) becomes 


f() = = | doer du f(u) e*, (13.4) 


—0oo 


This result is known as Fourier’s inversion theorem. 
From it we may define the Fourier transform of f(t) by 


Fi) = se | fioewas (13.5) 


and its inverse by 


fO= “Foye” de (13.6) 


1 
/2n —oO 
Including the constant 1/,/2x in the definition of f(o) (whose mathematical 
existence as T — 00 is assumed here without proof) is clearly arbitrary, the only 
requirement being that the product of the constants in (13.5) and (13.6) should 
equal 1/(2z). Our definition is chosen to be as symmetric as possible. 


> Find the Fourier transform of the exponential decay function f(t) = 0 for t < 0 and 


f(t)=Ae™ for t>0(A>0). 


Using the definition (13.5) and separating the integral into two parts, 


he 1 9 
= (O)e —iot dt+ he —At et dt 
V2n J—o 


‘ A e-Utio)t ae 
- DE - a 
A 


J2n(h + iw)’ 


which is the required transform. It is clear that the multiplicative constant A does not 
affect the form of the transform, merely its amplitude. This transform may be verified by 
resubstitution of the above result into (13.6) to recover f(t), but evaluation of the integral 
requires the use of complex-variable contour integration (chapter 24). < 


13.1.1 The uncertainty principle 


An important function that appears in many areas of physical science, either 
precisely or as an approximation to a physical situation, is the Gaussian or 
normal distribution. Its Fourier transform is of importance both in itself and also 
because, when interpreted statistically, it readily illustrates a form of uncertainty 
principle. 
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> Find the Fourier transform of the normalised Gaussian distribution 


o<t<o. 


peo exp ( <) 
win) \ 28)” 


This Gaussian distribution is centred on t = 0 and has a root mean square deviation 
At = t. (Any reader who is unfamiliar with this interpretation of the distribution should 
refer to chapter 30.) 

Using the definition (13.5), the Fourier transform of f(t) is given by 


2 


~ 1 ae 
f(o) wae Sa exp ( a) ex iot) dt 


= = 3 — exp {a [t? + 2c7iat + (1?iw)? — (77iw)’] \ dt, 


where the quantity —(t?iw)*/(2t?) has been both added and subtracted in the exponent 
in order to allow the factors involving the variable of integration t to be expressed as a 
complete square. Hence the expression can be written 


es sree { 1 se oe | \ 
f(o) Dn sar [_@ aa dts. 


The quantity inside the braces is the normalisation integral for the Gaussian and equals 
unity, although to show this strictly needs results from complex variable theory (chapter 24). 
That it is equal to unity can be made plausible by changing the variable to s = t + it?@ 
and assuming that the imaginary parts introduced into the integration path and limits 
(where the integrand goes rapidly to zero anyway) make no difference. 

We are left with the result that 


x 1 —10? 

f(@) = >= exp — , (13.7) 
/2n 2 

which is another Gaussian distribution, centred on zero and with a root mean square 

deviation Aw = 1/t. It is interesting to note, and an important property, that the Fourier 

transform of a Gaussian is another Gaussian. < 


In the above example the root mean square deviation in t was 1, and so it is 
seen that the deviations or ‘spreads’ in t and in @ are inversely related: 


Aw At = 1, 


independently of the value of t. In physical terms, the narrower in time is, say, an 
electrical impulse the greater the spread of frequency components it must contain. 
Similar physical statements are valid for other pairs of Fourier-related variables, 
such as spatial position and wave number. In an obvious notation, AKAx = 1 for 
a Gaussian wave packet. 

The uncertainty relations as usually expressed in quantum mechanics can be 
related to this if the de Broglie and Einstein relationships for momentum and 
energy are introduced; they are 


p=hk and E=ho. 


Here fi is Planck’s constant h divided by 27. In a quantum mechanics setting f(t) 
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is a wavefunction and the distribution of the wave intensity in time is given by 
|f/? (also a Gaussian). Similarly, the intensity distribution in frequency is given 
by If. These two distributions have respective root mean square deviations of 
t//2 and 1/(./21), giving, after incorporation of the above relations, 


AE At = h/2 and Ap Ax = h/2. 


The factors of 1/2 that appear are specific to the Gaussian form, but any 
distribution f(t) produces for the product AEAt a quantity Ah in which 4 is 
strictly positive (in fact, the Gaussian value of 1/2 is the minimum possible). 


13.1.2 Fraunhofer diffraction 


We take our final example of the Fourier transform from the field of optics. The 
pattern of transmitted light produced by a partially opaque (or phase-changing) 
object upon which a coherent beam of radiation falls is called a diffraction pattern 
and, in particular, when the cross-section of the object is small compared with 
the distance at which the light is observed the pattern is known as a Fraunhofer 
diffraction pattern. 

We will consider only the case in which the light is monochromatic with 
wavelength 1. The direction of the incident beam of light can then be described 
by the wave vector k; the magnitude of this vector is given by the wave number 
k = 2n/2 of the light. The essential quantity in a Fraunhofer diffraction pattern 
is the dependence of the observed amplitude (and hence intensity) on the angle 0 
between the viewing direction k’ and the direction k of the incident beam. This 
is entirely determined by the spatial distribution of the amplitude and phase of 
the light at the object, the transmitted intensity in a particular direction k’ being 
determined by the corresponding Fourier component of this spatial distribution. 

As an example, we take as an object a simple two-dimensional screen of width 
2Y on which light of wave number k is incident normally; see figure 13.2. We 
suppose that at the position (0, y) the amplitude of the transmitted light is f(y) 
per unit length in the y-direction (f(y) may be complex). The function f(y) is 
called an aperture function. Both the screen and beam are assumed infinite in the 
z-direction. 

Denoting the unit vectors in the x- and y- directions by i and j respectively, 
the total light amplitude at a position ro = xoi + yoj, with xo > 0, will be the 
superposition of all the (Huyghens’) wavelets originating from the various parts 
of the screen. For large ro (= |ro|), these can be treated as plane waves to give’ 


Y f(y)explik’ (ro — yD] , 
5 y. 
ay Ito — yjl 


A(ro) = (13.8) 


S This is the approach first used by Fresnel. For simplicity we have omitted from the integral a 
multiplicative inclination factor that depends on angle @ and decreases as 0 increases. 
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Figure 13.2 Diffraction grating of width 2Y with light of wavelength 22/k 
being diffracted through an angle 0. 


The factor exp[ik’ - (ro — yj)] represents the phase change undergone by the light 
in travelling from the point yj on the screen to the point ro, and the denominator 
represents the reduction in amplitude with distance. (Recall that the system is 
infinite in the z-direction and so the ‘spreading’ is effectively in two dimensions 
only.) 

If the medium is the same on both sides of the screen then k’ = k cos 0i+k sin 0j, 
and if ro >> Y then expression (13.8) can be approximated by 


A(ro) = <a [. f(y) exp(—iky sin 0) dy. (13.9) 


We have used that f(y) = 
The intensity in the direction @ is ee given by 


2n os 
1(0) = |AP = Filho (13.10) 
where q =ksin0. 


> Evaluate I(@) for an aperture consisting of two long slits each of width 2b whose centres 


are separated by a distance 2a, a > b; the slits are illuminated by light of wavelength 2. 


The aperture function is ae in figure 13.3. We first need to find f(a): 


—a+b a+b 
eax dx + =|, e4s dx. 
f= | a 


1 en tax —at+b iT en iax a+b 
+ |-5 
2a | Iq | -a-b 20 id lap 
= -1 [eta — e7ig(-a~b) 4 g—igtatb) _ eale—)) . 


iq./2n 
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f(y) 


x 


—a—b —a+b a—b “ at+b 


Figure 13.3 The aperture function f(y) for two wide slits. 


After some manipulation we obtain 
_ 4cosqasin qb 
qV2n ; 


Now applying (13.10), and remembering that g = (27 sin 0)/A, we find 


f(a) 


16 cos? gasin? qb 
r= em gaa 
qTo 


where ro is the distance from the centre of the aperture. < 


13.1.3 The Dirac 6-function 


Before going on to consider further properties of Fourier transforms we make a 
digression to discuss the Dirac 6-function and its relation to Fourier transforms. 
The 6-function is different from most functions encountered in the physical 
sciences but we will see that a rigorous mathematical definition exists; the utility 
of the 6-function will be demonstrated throughout the remainder of this chapter. 
It can be visualised as a very sharp narrow pulse (in space, time, density, etc.) 
which produces an integrated effect having a definite magnitude. The formal 
properties of the d-function may be summarised as follows. 
The Dirac 6-function has the property that 


O(t)=0 fort £0, (13.11) 

but its fundamental defining property is 
J teove —a)dt = f(a), (13.12) 
provided the range of integration includes the point t = a; otherwise the integral 
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equals zero. This leads immediately to two further useful results: 


b 
o(t)dt=1  forallasb>0 (13.13) 


—@ 


and 
[o-aar= 1, (13.14) 


provided the range of integration includes t = a. 
Equation (13.12) can be used to derive further useful properties of the Dirac 
6-function: 


o(t) = 6(—-0), (13.15) 
d(at) = alt (13.16) 
td(t) = 0. (13.17) 


> Prove that 6(bt) = 6(t)/|b|. 
Let us first consider the case where b > 0. It follows that 


| sepswnae= [" £(5) 0 = F100 = 5 [ stosinae, 


where we have made the substitution ¢’ = bt. But f(t) is arbitrary and so we immediately 
see that d(bt) = d(t)/b = d(t)/|b| for b > 0. 
Now consider the case where b = —c < 0. It follows that 


¥ f(t)6(bt) dt = [#4 d -} 3(t’) (=) = iZ “f (=) 3(t’) dt’ 


1 
= 210) = TA) =F [ floa[nae 


where we have made the substitution t’ = bt = —ct. But f(t) is arbitrary and so 
1 
6(bt) = —d(t), 
|b| 
for all b, which establishes the result. < 


Furthermore, by considering an integral of the form 


/ f(O5(h(t)) dt 


and making a change of variables to z = h(t), we may show that 


5 
d(h(t)) = X eae 17 (13.18) 


where the ¢; are those values of t for which h(t) = 0 and h'(t) stands for dh/dt. 
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The derivative of the delta function, 6’(t), is defined by 


0 oO 0 


| foo'ar = [fepaca]_ — fF Fanat 
=—f'(0), (13.19) 


and similarly for higher derivatives. 

For many practical purposes, effects that are not strictly described by a 6- 
function may be analysed as such, if they take place in an interval much shorter 
than the response interval of the system on which they act. For example, the 
idealised notion of an impulse of magnitude J applied at time tp can be represented 
by 


j(t) = JO(t — to). (13.20) 


Many physical situations are described by a 6-function in space rather than in 
time. Moreover, we often require the 6-function to be defined in more than one 
dimension. For example, the charge density of a point charge q at a point ro may 
be expressed as a three-dimensional 6-function 


P(t) = qo(r — ¥9) = qo(x — Xo)d(y — yo)0(z — Zo), (13.21) 


so that a discrete ‘quantum’ is expressed as if it were a continuous distribution. 
From (13.21) we see that (as expected) the total charge enclosed in a volume V 
is given by 


if ro lies in V 
[omar =f aoe 1) dV = ‘3 ie aes 
Vv Vv 


0 otherwise. 


Closely related to the Dirac 6-function is the Heaviside or unit step function 
H(t), for which 


1 fort>0, 
H(t) = (13.22) 
0 fort<0O. 


This function is clearly discontinuous at t = 0 and it is usual to take H(0) = 1/2. 
The Heaviside function is related to the delta function by 


H(t) = 4(t). (13.23) 
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> Prove relation (13.23). 


Considering the integral 


[soma =|pomo| —[ ronna 
=s00)- [roan 


Oe i) “= f00), 
0 


and comparing it with (13.12) when a = 0 immediately shows that H'(t) = d(t). < 


13.1.4 Relation of the 6-function to Fourier transforms 


In the previous section we introduced the Dirac 6-function as a way of repre- 
senting very sharp narrow pulses, but in no way related it to Fourier transforms. 
We now show that the 6-function can equally well be defined in a way that more 
naturally relates it to the Fourier transform. 

Referring back to the Fourier inversion theorem (13.4), we have 


fO=>5 ~ | doe" Pa du f(u) ei" 
= f du fn {= ~ fe io(t—u Naw}. 


Comparison of this with (13.12) shows that we may write the 6-function as 
6G =u) = =| gil) doy, (13.24) 
2n J_ 


Considered as a Fourier transform, this representation shows that a very 
narrow time peak at t = u results from the superposition of a complete spectrum 
of harmonic waves, all frequencies having the same amplitude and all waves being 
in phase at t = u. This suggests that the 6-function may also be represented as 
the limit of the transform of a uniform distribution of unit height as the width 
of this distribution becomes infinite. 

Consider the rectangular distribution of frequencies shown in figure 13.4(a). 
From (13.6), taking the inverse Fourier transform, 
1x eda 


fall) = =f 


20 sin Qt 


~ in Ot 


This function is illustrated in figure 13.4(b) and it is apparent that, for large Q, it 
becomes very large at t = 0 and also very narrow about t = 0, as we qualitatively 


(13.25) 
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folt) 
20 
(2n)'/2 \ 
fo 
1 
=O Q 0 7 \ | VW t 
Y z VU 
Seg = 


(a) (b) 


Figure 13.4 (a) A Fourier transform showing a rectangular distribution of 
frequencies between +Q; (b) the function of which it is the transform, which 
is proportional to ¢~! sin Qt. 


expect and require. We also note that, in the limit Q > «, fo(t), as defined by 
the inverse Fourier transform, tends to (2)'/*6(t) by virtue of (13.24). Hence we 
may conclude that the 6-function can also be represented by 

5(t) = Jim (=). (13.26) 


00 Tt 


Several other function representations are equally valid, e.g. the limiting cases of 
rectangular, triangular or Gaussian distributions; the only essential requirements 
are a knowledge of the area under such a curve and that undefined operations 
such as dividing by zero are not inadvertently carried out on the 6-function whilst 
some non-explicit representation is being employed. 

We also note that the Fourier transform definition of the delta function, (13.24), 
shows that the latter is real since 

0 
oO= =| e! da = 6(—t) = d(t). 
203} se 


Finally, the Fourier transform of a 6-function is simply 
~ 1 * ; 1 
(a) = — O(t)e dt = —=. (13.27) 


13.1.5 Properties of Fourier transforms 


Having considered the Dirac 6-function, we now return to our discussion of the 
properties of Fourier transforms. As we would expect, Fourier transforms have 
many properties analogous to those of Fourier series in respect of the connection 
between the transforms of related functions. Here we list these properties without 
proof; they can be verified by working from the definition of the transform. As 
previously, we denote the Fourier transform of f(t) by f(a) or F[f(0)]. 
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(i) Differentiation: 
Ff (t)| = iof(o). (13.28) 
This may be extended to higher derivatives, so that 
Ff" (0) =ioF[f'(0] = —0°f(o), 


and so on. 
(ii) Integration: 


| / f(s) i| = jlo) + 2ncd(c), (13.29) 


where the term 22cd(@) represents the Fourier transform of the constant 
of integration associated with the indefinite integral. 
(iii) Scaling: 


flat) = =F (2). (13.30) 
(iv) Translation: 
Flf(t ta] = ef (o). (13.31) 
(v) Exponential multiplication: 
F [e* f(t)| = fl + ia), (13.32) 


where « may be real, imaginary or complex. 


> Prove relation (13.28). 


Calculating the Fourier transform of f’(t) directly, we obtain 


Flf'O] = = : feat 


1 —iot _ ile ia . —iot 
= or e fo] 7 + Tin te ioe" f(t) dt 
= iof(o), 


if f(t) > 0 at t = +00, as it must since hes |f(t)| dt is finite. < 


To illustrate a use and also a proof of (13.32), let us consider an amplitude- 
modulated radio wave. Suppose a message to be broadcast is represented by f(t). 
The message can be added electronically to a constant signal a of magnitude 
such that a+ f(t) is never negative, and then the sum can be used to modulate 
the amplitude of a carrier signal of frequency we. Using a complex exponential 
notation, the transmitted amplitude is now 


g(t) =A [at f(t] ee. (13.33) 
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Ignoring in the present context the effect of the term Aaexp(imct), which gives a 
contribution to the transmitted spectrum only at w = wc, we obtain for the new 
spectrum 


2g(@) = A [" five iMet eTiwt dt 


“1 ge eae 
= pas - oy (13.34) 


which is simply a shift of the whole spectrum by the carrier frequency. The use 
of different carrier frequencies enables signals to be separated. 


13.1.6 Odd and even functions 


If f(t) is odd or even then we may derive alternative forms of Fourier’s inversion 
theorem, which lead to the definition of different transform pairs. Let us first 
consider an odd function f(t) = pia whose Fourier transform is given by 


1 


val 


= x i. f()(cos wt — isin wt) dt 
ape a f(d) sin ot dt, 


f(o) = “fle erat 


where in the last line we use the fact that f(t) and sinwt are odd, whereas cos wt 
is even. 
We note that f(—w) = —f(a), ie. f(@) is an odd function of w. Hence 


° pat, ies it _ di oS : 
fo= flo)e io = f f(@) sin wt dw 


[ « sino [ fluysinowdul 
0 


Thus we may define the Fourier sine transform pair for odd functions: 


flo y= 2 a f(t) sinat dt, (13.35) 
fo= (2 [3 fo ) sinwtda. (13.36) 


Note that although the Fourier sine transform pair was derived by considering 
an odd function f(t) defined over all t, the definitions (13.35) and (13.36) only 
require f(t) and f,(@) to be defined for positive t and w respectively. For an 
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8(y) | 


0 y 


Figure 13.5 Resolution functions: (a) ideal 6-function; (b) typical unbiased 
resolution; (c) and (d) biases tending to shift observations to higher values 
than the true one. 


even function, ic. one for which f(t) = f(—t), we can define the Fourier cosine 
transform pair in a similar way, but with sin wt replaced by cos at. 


13.1.7 Convolution and deconvolution 


It is apparent that any attempt to measure the value of a physical quantity is 
limited, to some extent, by the finite resolution of the measuring apparatus used. 
On the one hand, the physical quantity we wish to measure will be in general a 
function of an independent variable, x say, i.e. the true function to be measured 
takes the form f(x). On the other hand, the apparatus we are using does not give 
the true output value of the function; a resolution function g(y) is involved. By 
this we mean that the probability that an output value y = 0 will be recorded 
instead as being between y and y+dy is given by g(y) dy. Some possible resolution 
functions of this sort are shown in figure 13.5. To obtain good results we wish 
the resolution function to be as close to a 6-function as possible (case (a)). A 
typical piece of apparatus has a resolution function of finite width, although if 
it is accurate the mean is centred on the true value (case (b)). However, some 
apparatus may show a bias that tends to shift observations to higher or lower 
values than the true ones (cases (c) and (d)), thereby exhibiting systematic error. 

Given that the true distribution is f(x) and the resolution function of our 
measuring apparatus is g(y), we wish to calculate what the observed distribution 
h(z) will be. The symbols x, y and z all refer to the same physical variable (e.g. 
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—a a —bh b a a 


Figure 13.6 The convolution of two functions f(x) and g(y). 


length or angle), but are denoted differently because the variable appears in the 
analysis in three different roles. 

The probability that a true reading lying between x and x + dx, and so having 
probability f(x)dx of being selected by the experiment, will be moved by the 
instrumental resolution by an amount z — x into a small interval of width dz is 
g(z — x) dz. Hence the combined probability that the interval dx will give rise to 
an observation appearing in the interval dz is f(x) dx g(z —x) dz. Adding together 
the contributions from all values of x that can lead to an observation in the range 
z to z+ dz, we find that the observed distribution is given by 


h(z) = i f(x)g(z — x) dx. (13.37) 


The integral in (13.37) is called the convolution of the functions f and g and is 
often written f * g. The convolution defined above is commutative (f * g = g* f), 
associative and distributive. The observed distribution is thus the convolution of 
the true distribution and the experimental resolution function. The result will be 
that the observed distribution is broader and smoother than the true one and, if 
g(y) has a bias, the maxima will normally be displaced from their true positions. 
It is also obvious from (13.37) that if the resolution is the ideal 6-function, 
g(y) = 6(y) then A(z) = f(z) and the observed distribution is the true one. 

It is interesting to note, and a very important property, that the convolution of 
any function g(y) with a number of delta functions leaves a copy of g(y) at the 
position of each of the delta functions. 


> Find the convolution of the function f(x) = 6(x + a) + 6(x —a) with the function g(y) 
plotted in figure 13.6. 


Using the convolution integral (13.37) 
h(z) = a f0dg(z —x) dx = i) “[8(x + a) + 5(x— ale(z — x) dx 


= g(z +a) + g(z —4). 
This convolution h(z) is plotted in figure 13.6. < 


Let us now consider the Fourier transform of the convolution (13.37); this is 
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given by 


ik) = ene f(x)g(z —x)d x 


= [acto [ ae—me cae 


If we let u = z — x in the second integral we have 


h(k) = ae ° ax soot g(u) er) au 


“Tal. f(x)e™ dx [gue du 


= re x V2n f(k) x J2ng(k) = V2n f(K)R(K). (13.38) 
Hence the Fourier transform of a convolution f * g is equal to the product of the 
separate Fourier transforms multiplied by ./27; this result is called the convolution 
theorem. 
It may be proved similarly that the converse is also true, namely that the 
Fourier transform of the product f(x)g(x) is given by 


Ff (x)g(x)] = —=f(k) * Bk). (13.39) 


1 
Von 


> Find the Fourier transform of the function in figure 13.3 representing two wide slits by 
considering the Fourier transforms of (i) two 6-functions, at x = +a, (ii) a rectangular 
function of height 1 and width 2b centred on x = 0. 


(i) The Fourier transform of the two 6-functions is oe by 


x—a) Je dee 5(x +a) e74 dx 
/2n se fa —o 
_! (ete 4 elt) = Wien 


(ii) The Fourier transform of the broad slit is 


f(g) = 


so= hf ewar= 1 [2] 
ou 20 Jb V2n | —ig |_, 
-1 2singb 


<2 eiab elt) — 

iq./2n qvV2n 
We have already seen that the convolution of these functions is the required function 
representing two wide slits (see figure 13.6). So, using the convolution theorem, the Fourier 
transform of the convolution is ,/27 times the product of the individual transforms, ie. 


4cos qa sin gb/(q./27). This is, of course, the same result as that obtained in the example 
in subsection 13.1.2. < 
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The inverse of convolution, called deconvolution, allows us to find a true 
distribution f(x) given an observed distribution h(z) and a resolution function 


g(y). 


> An experimental quantity f(x) is measured using apparatus with a known resolution func- 
tion g(y) to give an observed distribution h(z). How may f(x) be extracted from the mea- 
sured distribution? 


From the convolution theorem (13.38), the Fourier transform of the measured distribution 
is 


hk) = 2x fik)g(k), 
from which we obtain 


ea 1 hk) 
k) = ———. 
IO) = em) 


Then on inverse Fourier transforming we find 


_ Lg [fit 
eae idl ae]. 


In words, to extract the true distribution, we divide the Fourier transform of the observed 
distribution by that of the resolution function for each value of k and then take the inverse 
Fourier transform of the function so generated. < 


This explicit method of extracting true distributions is straightforward for exact 
functions but, in practice, because of experimental and statistical uncertainties in 
the experimental data or because data over only a limited range are available, it 
is often not very precise, involving as it does three (numerical) transforms each 
requiring in principle an integral over an infinite range. 


13.1.8 Correlation functions and energy spectra 


The cross-correlation of two functions f and g is defined by 


C(z) = . f (x)g(x +z) dx. (13.40) 


Despite the formal similarity between (13.40) and the definition of the convolution 
in (13.37), the use and interpretation of the cross-correlation and of the convo- 
lution are very different; the cross-correlation provides a quantitative measure of 
the similarity of two functions f and g as one is displaced through a distance z 
relative to the other. The cross-correlation is often notated as C = f @g, and, like 
convolution, it is both associative and distributive. Unlike convolution, however, 
it is not commutative, in fact 


[f ® gl(z) = [g @ f]"(—z). (13.41) 
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» Prove the Wiener—Kinchin theorem, 


C(k) = V2a [fF (KN R(h). (13.42) 


Following a method similar to that for the convolution of f and g, let us consider the 
Fourier transform of (13.40): 


C(k) = ae dz ei? if. f(x dels-+ 2)ax} 
- —ikz 
=f. dx f "(x fe g(x+z)e ach. 


Making the substitution u = x +z in the second integral we obtain 


eye say ‘ " —ik(u—x) \ 
ere [ sme du 


= mht fiiooe™ ae g(u)e™" du 
=a (I x Vink) = V2m [FOR < 
Thus the Fourier transform of the cross-correlation of f and g is equal to 


the product of [f(k)I* and g(k) multiplied by ./2z. This a statement of the 
Wiener—Kinchin theorem. Similarly we can derive the converse theorem 


FIf (~e(o)] = al oz 


If we now consider the special case where g is taken to be equal to f in (13.40) 
then, writing the LHS as a(z), we have 


a(z) = [. f' (xf (x +z) dx; (13.43) 


this is called the auto-correlation function of f(x). Using the Wiener—Kinchin 
theorem (13.42) we see that 


a(z) = ak) el? dk 


oa 
ae a ry ay ikz 
=< | VR wriwe dk. 


so that a(z) is the inverse Fourier transform of ./27 IF(K)I?, which is in turn called 
the energy spectrum of f. 


13.1.9 Parseval’s theorem 


Using the results of the previous section we can immediately obtain Parseval’s 
theorem. The most general form of this (also called the multiplication theorem) 1s 
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obtained simply by noting from (13.42) that the cross-correlation (13.40) of two 
functions f and g can be written as 
C(z) = / f (x)g(x + 2) dx = i : [F(k))"g(k) el dk. (13.44) 


—00 


Then, setting z = 0 gives the multiplication theorem 


[ Foetsyax = | [Fe] Rak (13.45) 


Specialising further, by letting g = f, we derive the most common form of 
Parseval’s theorem, 


/ “Ifo dx = if “F(kK)P dk. (13.46) 
When f is a physical amplitude these integrals relate to the total intensity involved 
in some physical process. We have already met a form of Parseval’s theorem for 
Fourier series in chapter 12; it is in fact a special case of (13.46). 


> The displacement of a damped harmonic oscillator as a function of time is given by 


Fae i fort <0, 


e/*sin@ot for t> 0. 


Find the Fourier transform of this function and so give a physical interpretation of Parseval’s 
theorem. 


Using the usual definition for the Fourier transform we find 


0 eal 
f(@) = / 0x edt +f e~/* sin wot oe dt. 
- 0 
Writing sin mot as (e!o' — e~ 0") /2i we obtain 


f(a) =04 sf [e Mooi) = e Hoteo—i/) dt 
1 Jo 


= 1 1 
~ 2|@+@—i/t oW—a—i/t|’ 


which is the required Fourier transform. The physical interpretation of |f(c)|? is the energy 
content per unit frequency interval (i.e. the energy spectrum) whilst |f(t)|? is proportional to 
the sum of the kinetic and potential energies of the oscillator. Hence (to within a constant) 
Parseval’s theorem shows the equivalence of these two alternative specifications for the 
total energy. < 


13.1.10 Fourier transforms in higher dimensions 


The concept of the Fourier transform can be extended naturally to more than 
one dimension. For instance we may wish to find the spatial Fourier transform of 
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two- or three-dimensional functions of position. For example, in three dimensions 
we can define the Fourier transform of f(x, y,z) as 


Tks. ky, kz) = cus // f(x, ys z) ex ety y pike dx dy dz, (13.47) 
and its inverse as 
f(%Y,2) = rae / | [ilbobke) elkseikyy gikz= dk. dky dk. (13.48) 


Denoting the vector with components kx, ky, kz by k and that with components 
x, y, z by r, we can write the Fourier transform pair (13.47), (13.48) as 


ne 1 . 
f(k) = Gay / f(r) eT" Br, (13.49) 


1 Be 
f(r) = Ome / f (k) ec" ak. (13.50) 


From these relations we may deduce that the three-dimensional Dirac 6-function 
can be written as 


o(r) = 


1 ik-r 33 
a |e ak. (13.51) 


Similar relations to (13.49), (13.50) and (13.51) exist for spaces of other dimen- 
sionalities. 


In three-dimensional space a function f(r) possesses spherical symmetry, so that f(r) = 
f(r). Find the Fourier transform of f(r) as a one-dimensional integral. 


Let us choose spherical polar coordinates in which the vector k of the Fourier transform 
lies along the polar axis (0 = 0). This we can do since f(r) is spherically symmetric. We 
then have 

@r=r'sinOdrd0d® and k-r=krcosd, 


where k = |k|. The Fourier transform is then given by 


ws 1 —ik-r 
7) = Gos [five as 


1 ” ge Dee —ikr cos 0 
-oon | ar | ao | d@ f(r)r° sin de 


es a | «dt Inf (Hr : dO sin 0 eros? 
0 


0 
The integral over 0 may be straightforwardly evaluated by noting that 


a (ei cos %) 


de 


= ikrsin0 e7ikr cos 0 


Therefore 


Fk 1 ro) as j ewikr cos 07 9= 
= r 2nf(r)r- | ——— 
We) (2n)*/ [ oo) ikr ies 


oa aan [ Anr’f(r) (==) dr. < 
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A similar result may be obtained for two-dimensional Fourier transforms in 
which f(r) = f(p), ie. f(r) is independent of azimuthal angle ¢. In this case, using 
the integral representation of the Bessel function Jo(x) given at the very end of 
subsection 18.5.3, we find 


os 1 
Fk) = = | 2npf(p)Jo(kp) dp. (13.52) 


13.2 Laplace transforms 


Often we are interested in functions f(t) for which the Fourier transform does not 
exist because f 4 0 as t > o, and so the integral defining f does not converge. 
This would be the case for the function f(t) = t, which does not possess a Fourier 
transform. Furthermore, we might be interested in a given function only for t > 0, 
for example when we are given the value at t = O in an initial-value problem. 
This leads us to consider the Laplace transform, f(s) or “[f(t)], of f(t), which 
is defined by 


fls)= | * f(t)e dt, (13.53) 


provided that the integral exists. We assume here that s is real, but complex values 
would have to be considered in a more detailed study. In practice, for a given 
function f(t) there will be some real number so such that the integral in (13.53) 
exists for s > so but diverges for s < so. 

Through (13.53) we define a linear transformation & that converts functions 
of the variable t to functions of a new variable s: 


L [af i(t) + bf2(t)] = aF [filo] + bF [fr(0)] = afi(s) + bfr(s). (13.54) 


> Find the Laplace transforms of the functions (i) f(t) = 1, (ii) f(t) = e”, (iti) f(t) = t", 
OTT — 10 mle ares 


(i) By direct application of the definition of a Laplace transform (13.53), we find 
Soaeat -1_,J° 1 : 
eiy- f edr= |e ‘ =-, ifs>0, 
0 s o Ss 


where the restriction s > 0 is required for the integral to exist. 
(ii) Again using (13.53) directly, we find 


fis)= i “ele dt = ip = elas dt 
0 0 


(a—s)t 17% 1 
e : 
a = ifs>a. 
0 


a—s S—a 
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(iii) Once again using the definition (13.53) we have 


fas) = fo treat 
0 


Integrating by parts we find 
7” —tlet 99 oe) 
asy= |=] et fortes a 
s 0 


0 Ss 


= 0+ =Fr-1(3), ifs>0. 


We now have a recursion relation between successive transforms and by calculating 
fo we can infer f,, f2, etc. Since t° = 1, (i) above gives 


ee. Gee 13.55 
S 


and 
fils) = = foals) = = wy fil8) = a ifs >0. 

Thus, in each case (i)-(iii), direct application of the definition of the Laplace transform 
(13.53) yields the required result. < 

Unlike that for the Fourier transform, the inversion of the Laplace transform 
is not an easy operation to perform, since an explicit formula for f(t), given f(s), 
is not straightforwardly obtained from (13.53). The general method for obtaining 
an inverse Laplace transform makes use of complex variable theory and is not 
discussed until chapter 25. However, progress can be made without having to find 
an explicit inverse, since we can prepare from (13.53) a ‘dictionary’ of the Laplace 
transforms of common functions and, when faced with an inversion to carry out, 
hope to find the given transform (together with its parent function) in the listing. 
Such a list is given in table 13.1. 

When finding inverse Laplace transforms using table 13.1, it is useful to note 
that for all practical purposes the inverse Laplace transform is unique’ and linear 
so that 


ZL" [afi(s) + bfa(s)] = afi(t) + bfr(0). (13.56) 


In many practical problems the method of partial fractions can be useful in 
producing an expression from which the inverse Laplace transform can be found. 


> Using table 13.1 find f(t) if 


S This is not strictly true, since two functions can differ from one another at a finite number of 
isolated points but have the same Laplace transform. 
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fp) f(s) So 
c c/s 0 
ct" cnt /s"*1 0 
sin bt b/(s? +b’) 0 
cos bt s/(s? +b’) 0 
et 1/(s—a) a 
tet n!/(s—a)"*! a 
sinh at a/(s? — a’) lal 
cosh at s/(s? — a’) a 
e“ sin bt b/[(s — a)? + b?] a 
e" cos bt (s—a)/[(s — a)? + 7] a 
p/2 5(1/s°)'? 0 
pl/2 (n/s)/? 0 
O(t — to) eo 0 
H(t — to) = {0 ee e*/s 0 
0 fort < to 


Table 13.1 Standard Laplace transforms. The transforms are valid for s > so. 


Comparing this with the standard Laplace transforms in table 13.1, we find that the inverse 
transform of 3/s is 3 for s > 0 and the inverse transform of 2/(s+ 1) is 2e~ for s > —1, 
and so 


f(t)=3-2e', ifs>0.< 


13.2.1 Laplace transforms of derivatives and integrals 


One of the main uses of Laplace transforms is in solving differential equations. 
Differential equations are the subject of the next six chapters and we will return 
to the application of Laplace transforms to their solution in chapter 15. In 
the meantime we will derive the required results, i.e. the Laplace transforms of 
derivatives. 

The Laplace transform of the first derivative of f(t) is given by 


df] f° df —. 
L | - | a dt 
= [fe] +sf f(ne™ dt 
= —f(0) + sf(s), 


The evaluation relies on integration by parts and higher-order derivatives may 
be found in a similar manner. 


for s > 0. (13.57) 
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> Find the Laplace transform of df /dt?. 


Using the definition of the Laplace transform and integrating by parts we obtain 
a f oO a f 
L|\— |= i) —e dt 
dt? 9 at . 


eal’ a5 f° Hes 
= Si r std 
ae ) ae di ‘ 


= 10) + s[sf(s) — f(0)], for s > 0, 


where (13.57) has been substituted for the integral. This can be written more neatly as 


= Fa sf (s)— sf(0) So), fors>0. < 


In general the Laplace transform of the nth derivative is given by 


a"f ene n—ly¢ n24f 
z|Z|-s —s"“f(0)—s 71) 


7(0), fors>0. 
(13.58) 


We now turn to integration, which is much more straightforward. From the 
definition (13.53), 


L if fwd -{ dt e~* [ seve 
Te ocaeet fs en i eee 
= x6 [ tonaul. +f se f(t) dt. 


The first term on the RHS vanishes at both limits, and so 


L | fluydul = = (fl. (13.59) 


13.2.2 Other properties of Laplace transforms 


From table 13.1 it will be apparent that multiplying a function f(t) by e“ has the 
effect on its transform that s is replaced by s — a. This is easily proved generally: 


2 le"f@) = ) f(te"e™ dt 
0 
= } : f(tle & dt 
0 
= f(s—a). (13.60) 
As it were, multiplying f(t) by e“ moves the origin of s by an amount a. 
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We may now consider the effect of multiplying the Laplace transform f(s) by 
e°’ (b > 0). From the definition (13.53), 


e f(s) = [ Te sng(a at 
0 


= [ e “f(z —b)dz, 
0 


on putting t+ b = z. Thus e~’’f(s) is the Laplace transform of a function g(t) 
defined by 


Oe 0 for0<t<b, 
aimee Cfcee Mec oe) 
In other words, the function f has been translated to ‘later’ t (larger values of ft) 
by an amount b. 


Further properties of Laplace transforms can be proved in similar ways and 
are listed below. 


( £ pa) = =F (5), (13.61) 
(ii) Llt'fol = pes a ome (13.62) 
(iii) g ca ds i apie (13.63) 


provided lim,_,o[ f(t)/t] exists. 


Related results may be easily proved. 


> Find an expression for the Laplace transform of t d’f /dt?. 


From the definition of the Laplace transform we have 


af a day 
2 -| e tra at 


d rz if 
as [s°f(s) — sf) — f'(0)] 
2 SEE pes 
=-s° 5 — 2sf + f(0). < 
Finally we mention the convolution theorem for Laplace transforms (which is 


analogous to that for Fourier transforms discussed in subsection 13.1.7). If the 
functions f and g have Laplace transforms f(s) and 3(s) then 


L | f(u)g(t — u) a = f(s)z(s), (13.64) 
0 
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wr 


(a) (b) 


Figure 13.7 Two representations of the Laplace transform convolution (see 
text). 


where the integral in the brackets on the LHS is the convolution of f and g, 
denoted by f * g. As in the case of Fourier transforms, the convolution defined 
above is commutative, ie. f * g = g * f, and is associative and distributive. From 
(13.64) we also see that 


£" [F(s)2(s)] = [ felt —u) du =f *g. 


> Prove the convolution theorem (13.64) for Laplace transforms. 
From the definition (13.64), 


Fi9a(s) = fi “ef (u) du i, “eg(v) do 


=e io z 5 a S(u+e) 5 
[wf dve f(wg(v). 


Now letting u + v = t changes the limits on the integrals, with the result that 


Fi9a(s) = | “du f(u) : “digit we. 


As shown in figure 13.7(a) the shaded area of integration may be considered as the sum 
of vertical strips. However, we may instead integrate over this area by summing over 
horizontal strips as shown in figure 13.7(b). Then the integral can be written as 


F(s)g(s) = i du f(u) [ “ dtg(t—wye™ 


= i dte™ { ff renee udu} 


= lf fluygtt— ud _< 
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The properties of the Laplace transform derived in this section can sometimes 
be useful in finding the Laplace transforms of particular functions. 


> Find the Laplace transform of f(t) = tsin bt. 


Although we could calculate the Laplace transform directly, we can use (13.62) to give 


7 d ; d b 2bs 
f(s) = (“NF [sin bt] F (5 a z) @+RP’ fors>0. <4 


13.3 Concluding remarks 


In this chapter we have discussed Fourier and Laplace transforms in some detail. 
Both are examples of integral transforms, which can be considered in a more 
general context. 

A general integral transform of a function f(t) takes the form 


b 
F(a) = / K(a,t)f(t) dt, (13.65) 


where F(a) is the transform of f(t) with respect to the kernel K(a,t), and « is 
the transform variable. For example, in the Laplace transform case K(s, t) = e™, 
a=0,b=0. 

Very often the inverse transform can also be written straightforwardly and 
we obtain a transform pair similar to that encountered in Fourier transforms. 
Examples of such pairs are 


(i) the Hankel transform 
F(k) = ifs f (x)In(kx)x dx, 
0 
fic) = [Fda dbo dk, 
0 


where the J, are Bessel functions of order n, and 
(ii) the Mellin transform 


F(z) = ie 7" f(t) dt, 


ae cae 
fo= al t * F(z) dz. 
TL J ico 
Although we do not have the space to discuss their general properties, the 
reader should at least be aware of this wider class of integral transforms. 
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13.2 


13.3 
13.4 


13.6 


13.4 Exercises 
Find the Fourier transform of the function f(t) = exp(—|t}). 


(a) By applying Fourier’s inversion theorem prove that 


= t 
1 expen | cos @ - 
2 0 


1+? 


(b) By making the substitution w = tan 0, demonstrate the validity of Parseval’s 
theorem for this function. 


Use the general definition and properties of Fourier transforms to show the 
following. 


(a) If f(x) is periodic with period a then f(k) =0, unless ka = 2nn for integer n. 
(b) The Fourier transform of tf(t) is idf(@)/do. 
(c) The Fourier transform of f(mt +c) is 


calla (2) 
m m/~ 


Find the Fourier transform of H(x—a)e~*, where H(x) is the Heaviside function. 
Prove that the Fourier transform of the function f(t) defined in the tf-plane by 
straight-line segments joining (—T,0) to (0,1) to (T,0), with f(t) = 0 outside 
|t| < T, is 


where sinc x is defined as (sin x)/x. 

Use the general properties of Fourier transforms to determine the transforms 
of the following functions, graphically defined by straight-line segments and equal 
to zero outside the ranges specified: 


(a) (0,0) to (0.5, 1) to (1,0) to (2,2) to (3,0) to (4.5,3) to (6,0); 
) (2,0) to (—1,2) to (1,2) to (2,0): 

(c) (0,0) to (0,1) to (1,2) to (1,0) to (2,1) to (2,0). 

By 


taking the Fourier transform of the equation 


e 


Pp 


—S Kb = f(s), 
show that its solution, ¢(x), can be written as 
a eal oikx fF 

(x) = i paphie) dk, 


J2n J e+ K? 
where f(k) is the Fourier transform of f(x). 

By differentiating the definition of the Fourier sine transform f,(«) of the function 
f(t) =t-'? with respect to w, and then integrating the resulting expression by 
parts, find an elementary differential equation satisfied by f,(@). Hence show that 
this function is its own Fourier sine transform, ie. f,(@) = Af(@), where A is a 
constant. Show that it is also its own Fourier cosine transform. Assume that the 
limit as x > oo of x!/? sinax can be taken as zero. 

Find the Fourier transform of the unit rectangular distribution 


fer={ ls 


otherwise. 


13.4 EXERCISES 


13.8 


13.9 


Determine the convolution of f with itself and, without further integration, 
deduce its transform. Deduce that 


© sin? w 
— da = 7, 
_~» OO 


00 


oO gid 
sin” @ 2n 
He oF do = a 


00 


Calculate the Fraunhofer spectrum produced by a diffraction grating, uniformly 
illuminated by light of wavelength 22/k, as follows. Consider a grating with 4N 
equal strips each of width a and alternately opaque and transparent. The aperture 
function is then 


ae A for (2n+l)a<y<(2n+2)a, -N<n<N, 
yy 0 otherwise. 


(a) Show, for diffraction at angle @ to the normal to the grating, that the required 
Fourier transform can be written 
N-1 2a 
f(a) = (2n)71/? is exp(—2iarg) f Aexp(—iqu) du, 
r=—N $ 
where q =k sin 0. 
(b) Evaluate the integral and sum to show that 


& = ; Asin(2qaN 
F(a) = (2ny¥ exp(—iga/2) 2H) 
qcos(qa/2) 
and hence that the intensity distribution [(@) in the spectrum is proportional 
to 
sin? (2qaN) 
g? cos*(qa/2)" 


(c) For large values of N, the numerator in the above expression has very closely 
spaced maxima and minima as a function of 6 and effectively takes its mean 
value, 1/2, giving a low-intensity background. Much more significant peaks 
in I(@) occur when 6 = 0 or the cosine term in the denominator vanishes. 
Show that the corresponding values of |f(q)| are 


2aNA bd 4aNA 
(2n)!/2 (22)'/2(2m + 1)x’ 


Note that the constructive interference makes the maxima in I(0) oc N’, not 
N. Of course, observable maxima only occur for 0 < 0 < 2/2. 


with m integral. 


By finding the complex Fourier series for its LHS show that either side of the 
equation 

= 1 = —2nnit/T 

2 d(t+nT) = = S e 


can represent a periodic train of impulses. By expressing the function f(t + nX), 
in which X is a constant, in terms of the Fourier transform f(@) of f(t), show 
that 


> f(t+nx) = ae > i(=) eumnit/X 


n=—H n=—oo 


This result is known as the Poisson summation formula. 
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13.10 


13.11 


13.12 


13.13 


In many applications in which the frequency spectrum of an analogue signal is 
required, the best that can be done is to sample the signal f(t) a finite number of 
times at fixed intervals, and then use a discrete Fourier transform F, to estimate 
discrete points on the (true) frequency spectrum f(a). 


(a) By an argument that is essentially the converse of that given in section 13.1, 
show that, if N samples f,, beginning at t = 0 and spaced t apart, are taken, 
then f(22k/(Nt)) = Ft where 


en 2amnki/N 


, Xa 
k= n 
(b) For the function f(t) defined by 


w= {i for0<t<1, 


0 otherwise, 


from which eight samples are drawn at intervals of t = 0.25, find a formula 
for |F,| and evaluate it for k = 0,1,...,7. 

(c) Find the exact frequency spectrum of f(t) and compare the actual and 
estimated values of /2n\f (@)| at wo = ka for k = 0,1,...,7. Note the 
relatively good agreement for k < 4 and the lack of agreement for larger 
values of k. 


For a function f(t) that is non-zero only in the range |t| < T/2, the full frequency 
spectrum f(@) can be constructed, in principle exactly, from values at discrete 
sample points w = n(27/T). Prove this as follows. 


(a) Show that the coefficients of a complex Fourier series representation of f(t) 
with period T can be written as 
VJ2n ~ ( 2mn 
Ch = opt (+) : 


(b) Use this result to represent f(t) as an infinite sum in the defining integral for 
f(@), and hence show that 


f(o)= > f (=) sinc G - ) : 


n=—0O 
where sinc x is defined as (sin x)/x. 


A signal obtained by sampling a function x(t) at regular intervals T is passed 
through an electronic filter, whose response g(t) to a unit 6-function input is 
represented in a tg-plot by straight lines joining (0,0) to (T,1/T) to (2T,0) and 
is zero for all other values of t. The output of the filter is the convolution of the 
input, >”, x(t)6(t — nT), with g(t). 

Using the convolution theorem, and the result given in exercise 13.4, show that 
the output of the filter can be written 


y= D> xr) [ 


n=—o0 = 


(oa) 


sinice ( oT ) e-ielln+)T—1 Jay 
5a : 


Find the Fourier transform specified in part (a) and then use it to answer part 


(b). 
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13.14 


13.15 


13.16 


13.17 


(a) Find the Fourier transform of 


esinpt t>0, 
» i= 
f(y, p, t) re t<0, 


where (> 0) and p are constant parameters. 
(b) The current I(t) flowing through a certain system is related to the applied 
voltage V(t) by the equation 
I(t) = K(t—u)V(u) du, 
where 
K(t) = aif (v1, p17) + af (y2, P27). 


The function f(y, p,t) is as given in (a) and all the a;,y; (> 0) and p; are fixed 
parameters. By considering the Fourier transform of I(t), find the relationship 
that must hold between a, and a) if the total net charge Q passed through 
the system (over a very long time) is to be zero for an arbitrary applied 
voltage. 


Prove the equality 


a) 1 ce a 
e" sin? at dt = — Fa do. 
0 t Jo 4at*+o 


A linear amplifier produces an output that is the convolution of its input and its 
response function. The Fourier transform of the response function for a particular 
amplifier is 
is io 
K(o) = ———.. 
J2n(a + iw)? 


Determine the time variation of its output g(t) when its input is the Heaviside 
step function. (Consider the Fourier transform of a decaying exponential function 
and the result of exercise 13.2(b).) 

In quantum mechanics, two equal-mass particles having momenta p; = hk; and 
energies E; = ha; and represented by plane wavefunctions @; = exp|i(k;-rj;—«,t)], 
j = 1,2, interact through a potential V = V(|r, —r|). In first-order perturbation 
theory the probability of scattering to a state with momenta and energies pj, E; 
is determined by the modulus squared of the quantity 


M= [fv Vi dry dry dt. 


The initial state, y;, is @1¢2 and the final state, wr, is $5. 


(a) By writing r) +r = 2R and r; —rm =r and assuming that dr, dr. = dRar, 
show that M can be written as the product of three one-dimensional integrals. 

(b) From two of the integrals deduce energy and momentum conservation in the 
form of 6-functions. > 

(c) Show that M is proportional to the Fourier transform of V, ie. to V(k) 
where 2hk = (p2 — pi) — (ps — p}) or, alternatively, Ak = p; — pr. 


For some ion—atom scattering processes, the potential V of the previous exercise 
may be approximated by V = |r; — r|~! exp(—p|r1 — r2|). Show, using the result 
of the worked example in subsection 13.1.10, that the probability that the ion 
will scatter from, say, p; to p} is proportional to («7 +k?)-°, where k = |k| and k 
is as given in part (c) of that exercise. 
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13.18 


13.19 


13.20 


13.21 


13.22 


13.23 


The equivalent duration and bandwidth, T, and B., of a signal x(t) are defined 
in terms of the latter and its Fourier transform X(w) by 


1 ice) 
T= af. x(t) dt, 


1 ee 
Be = an |. X(w) do, 


where neither x(0) nor X(0) is zero. Show that the product T,B, = 27 (this is a 
form of uncertainty principle), and find the equivalent bandwidth of the signal 


x(t) = exp(—|t|/T). 


For this signal, determine the fraction of the total energy that lies in the frequency 
range |w| < B./4. You will need the indefinite integral with respect to x of 
(a? + x*)-?, which is 

x 1. 4x 
2a?(a2 + x?) Hs 2a3 an a 


Calculate directly the auto-correlation function a(z) for the product f(t) of the 
exponential decay distribution and the Heaviside step function, 


f(t)= fe" H(t) 


Use the Fourier transform and energy spectrum of f(t) to deduce that 


al elez Tt —ilel 
Dow dwo= se. 
-o V+o 2 


Prove that the cross-correlation C(z) of the Gaussian and Lorentzian distributions 


1 ? a 1 
th= = t)=(-—-) —— 
nO Eo ( =): e (=) P +a? 
has as its Fourier transform the function 


Cw 


1 
oe ( 5) 


1 a — 2? az 
C(z) ie exp ( = ) cos (S ) 3 
Prove the expressions given in table 13.1 for the Laplace transforms of t— 
t'/?, by setting x* = ts in the result 


/ exp(—x’) dx = 4.,/T. 
0 
Find the functions y(t) whose Laplace transforms are the following: 


(a) 1/(s?—s—2); 

(b) 2s/[(s + 1)(s* +4]; 

(c) eC OMO/[(s+y)P +b]. 

Use the properties of Laplace transforms to prove the following without evaluat- 
ing any Laplace integrals explicitly: 

(a) & [15/2] = Bas; 

(b) & [(sinhat)/t] = 41n[(st+a)/(s—a)], 5 > lal; 


) exp(—al)). 


Hence show that 


1/2 and 
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13.24 


13.25 


13.26 


(c) Y [sinh atcos bt] = a(s? — a? + b*)[(s — a)? +b] [(s +a? + bY. 


Find the solution (the so-called impulse response or Green’s function) of the 
equation 
dx 


T 
dt 


+x = d(t) 
by proceeding as follows. 
(a) Show by substitution that 
x(t) = A(1 — ee!" ) H(t) 
is a solution, for which x(0) = 0, of 


dx 
dt 


where H(t) is the Heaviside step function. 

(b) Construct the solution when the RHS of (*) is replaced by AH(t — 1), with 
dx/dt = x = 0 for t < t, and hence find the solution when the RHS is a 
rectangular pulse of duration t. 

(c) By setting A = 1/t and taking the limit as t > 0, show that the impulse 
response is x(t) = T~!e-/", 

(d) Obtain the same result much more directly by taking the Laplace transform 
of each term in the original equation, solving the resulting algebraic equation 
and then using the entries in table 13.1. 


T— +x =AH(0), (*) 


This exercise is concerned with the limiting behaviour of Laplace transforms. 


(a) If f(t) = A+ g(t), where A is a constant and the indefinite integral of g(t) is 
bounded as its upper limit tends to 00, show that 


lim sf(s) = A. 
(b) For t > 0, the function y(t) obeys the differential equation 


‘ig dy 
= +a + by = ccos’ wt, 
where a, b and c are positive constants. Find j(s) and show that sj(s) > c/2b 
as s — 0. Interpret the result in the t-domain. 
By writing f(x) as an integral involving the 6-function 6(€ — x) and taking the 
Laplace transforms of both sides, show that the transform of the solution of the 
equation 
dé 
— —y =f) 


dx* 


for which y and its first three derivatives vanish at x = 0 can be written as 


16) = f° tga. 


Use the properties of Laplace transforms and the entries in table 13.1 to show 
that 


vos) = 5 fH Tsinbos— &) —sin(x— 2) 8. 
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13.27 


13.28 


13.3 
13.5 


13.11 


13.13 


13.15 


The function f,(x) is defined as unity for 0 < x < a and zero otherwise. Find its 
Laplace transform f(s) and deduce that the transform of xfa(x) is 


es [1-(L+as)e™]. 


Write f,(x) in terms of Heaviside functions and hence obtain an explicit expres- 
sion for 


go(x) = | falyfolx — y) dy. 


Use the expression to write ,(s) in terms of the functions f,(s) and fr9( oe ae 
their derivatives, and hence show that 2,(s) is equal to the square of f,(s), in 
accordance with the convolution theorem. 

Show that the Laplace transform of f(t —a)H(t —a), where a > 0, is e~“ f(s) and 
that, if g(t) is a periodic function of period T, g(s) can be written as 


1 T 
{Ser | e o(t) dt. 
= 0 


(a) Sketch the periodic function defined in 0 <t< T by 
= 2t/T 0<t<T/2, 
8) =) 1 -t/T) T/2<t<T, 


and, using the previous result, find its Laplace transform. 
(b) Show, by sketching it, that 


Som) + 20M — 5nT)H(t— 5nT)] 


n=1 


is another representation of g(t) and hence derive the relationship 


tanhx =14+25°(-1)"e™. 


n=1 


13.5 Hints and answers 


Note that the integrand has different analytic forms for t < 0 and t > 0. 
(2/n)'/?(1 +o is 
(1//2n)[(b /(b? + k)Jeu b+ik) 


Use or derive ie = —k?¢(k) to obtain an algebraic equation for $(k) and then 
use the Fourier inversion formula. 

(2//2n)(sin w/o). 

The convolution is 2—|t| for |t] < 2, zero otherwise. Use the convolution theorem. 
(4/./2n)(sin* w/o”). 

Apply Parseval’s theorem to f and to f * f. 

The Fourier coefficient is T~!, independent of n. Make the changes of variables 
t—> o,n—-—nand T > 27/X and apply the translation theorem. 


(b) Recall that the infinite integral involved in defining f(@) has a non-zero 
integrand only in |t| < T/2. 

a) (1//2n){p/[(y + io)? + p"}}.- 

(b) Show that Q = y2n1(0) and use the convolution theorem. The required 
relationship is api /(y{ + pt) + a2p2/(93 + pz) = 0 

3(w) = 1/[V2n(a + iw)?], leading to g(t) = te. 
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13.17 
13.19 


13.21 
13.23 


13.25 


13.27 


Vk) oc [—2n/(ik)] [fexpl—(u — kyr] — expl-(u + ik)r]} dr. 

Note that the lower limit in the calculation of a(z) is 0, for z > 0, and |z|, for 
z <0. Auto-correlation a(z) = [(1/(2A3)] exp(—A|z]). 

Prove the result for t!/? by integrating that for t~'/? by parts. 

(a) Use (13.62) with n= 2 on Y [Vt] ; (b) use (13.63); 

(c) consider Y [exp(tat) cos bt] and use the translation property, subsection 13.2.2. 
(a) Note that | lim f g(t)e~ dt| < | lim [ g(t) del. 

(b) (s? + as + b)H(s) = fe(s? + 207)/[s(s? + 40”)]} + (a+ s)y(0) + (0). 

For this damped system, at large t (corresponding to s > 0) rates of change 
are negligible and the equation reduces to by = ccos’ wt. The average value of 
cos? wt is 5. 
s-'[1 — exp(—sa)]; ga(x) = x for 0 < x < a, g(x) = 2a—x for a < x < 2a, 
2a(x) = 0 otherwise. 
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First-order ordinary differential 
equations 


Differential equations are the group of equations that contain derivatives. Chap- 
ters 14-21 discuss a variety of differential equations, starting in this chapter and 
the next with those ordinary differential equations (ODEs) that have closed-form 
solutions. As its name suggests, an ODE contains only ordinary derivatives (no 
partial derivatives) and describes the relationship between these derivatives of 
the dependent variable, usually called y, with respect to the independent variable, 
usually called x. The solution to such an ODE is therefore a function of x and 
is written y(x). For an ODE to have a closed-form solution, it must be possible 
to express y(x) in terms of the standard elementary functions such as exp x, Inx, 
sin x etc. The solutions of some differential equations cannot, however, be written 
in closed form, but only as an infinite series; these are discussed in chapter 16. 

Ordinary differential equations may be separated conveniently into differ- 
ent categories according to their general characteristics. The primary grouping 
adopted here is by the order of the equation. The order of an ODE is simply the 
order of the highest derivative it contains. Thus equations containing dy/dx, but 
no higher derivatives, are called first order, those containing d?y/dx? are called 
second order and so on. In this chapter we consider first-order equations, and in 
the next, second- and higher-order equations. 

Ordinary differential equations may be classified further according to degree. 
The degree of an ODE is the power to which the highest-order derivative is 
raised, after the equation has been rationalised to contain only integer powers of 
derivatives. Hence the ODE 

a dy \3”? 

s+x(2) +x’y =0, 
is of third order and second degree, since after rationalisation it contains the term 
(@y/dx?). 

The general solution to an ODE is the most general function y(x) that satisfies 
the equation; it will contain constants of integration which may be determined by 
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14.1 GENERAL FORM OF SOLUTION 


the application of some suitable boundary conditions. For example, we may be 
told that for a certain first-order differential equation, the solution y(x) is equal to 
zero when the parameter x is equal to unity; this allows us to determine the value 
of the constant of integration. The general solutions to nth-order ODEs, which 
are considered in detail in the next chapter, will contain n (essential) arbitrary 
constants of integration and therefore we will need n boundary conditions if these 
constants are to be determined (see section 14.1). When the boundary conditions 
have been applied, and the constants found, we are left with a particular solution 
to the ODE, which obeys the given boundary conditions. Some ODEs of degree 
greater than unity also possess singular solutions, which are solutions that contain 
no arbitrary constants and cannot be found from the general solution; singular 
solutions are discussed in more detail in section 14.3. When any solution to an 
ODE has been found, it is always possible to check its validity by substitution 
into the original equation and verification that any given boundary conditions 
are met. 

In this chapter, firstly we discuss various types of first-degree ODE and then go 
on to examine those higher-degree equations that can be solved in closed form. 
At the outset, however, we discuss the general form of the solutions of ODEs; 
this discussion is relevant to both first- and higher-order ODEs. 


14.1 General form of solution 


It is helpful when considering the general form of the solution of an ODE to 
consider the inverse process, namely that of obtaining an ODE from a given 
group of functions, each one of which is a solution of the ODE. Suppose the 
members of the group can be written as 


y = f(x, 41, a2,...,dn), (14.1) 


each member being specified by a different set of values of the parameters a;. For 
example, consider the group of functions 


y =a, sinx + a) cos x; (14.2) 


here n = 2. 

Since an ODE is required for which any of the group is a solution, it clearly 
must not contain any of the a;. As there are n of the a; in expression (14.1), we 
must obtain n+ 1 equations involving them in order that, by elimination, we can 
obtain one final equation without them. 

Initially we have only (14.1), but if this is differentiated n times, a total of n+1 
equations is obtained from which (in principle) all the a; can be eliminated, to 
give one ODE satisfied by all the group. As a result of the n differentiations, 
d"y/dx" will be present in one of the n+ 1 equations and hence in the final 
equation, which will therefore be of nth order. 
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In the case of (14.2), we have 


[n= a, COS X — ay sin x, 
dx 
@y 
dx? 
Here the elimination of a, and ap is trivial (because of the similarity of the forms 
of y and d’?y/dx?), resulting in 


= —a, sin xX — a) COSx. 


a second-order equation. 

Thus, to summarise, a group of functions (14.1) with n parameters satisfies an 
nth-order ODE in general (although in some degenerate cases an ODE of less 
than nth order is obtained). The intuitive converse of this is that the general 
solution of an nth-order ODE contains n arbitrary parameters (constants); for 
our purposes, this will be assumed to be valid although a totally general proof is 
difficult. 

As mentioned earlier, external factors affect a system described by an ODE, 
by fixing the values of the dependent variables for particular values of the 
independent ones. These externally imposed (or boundary) conditions on the 
solution are thus the means of determining the parameters and so of specifying 
precisely which function is the required solution. It is apparent that the number 
of boundary conditions should match the number of parameters and hence the 
order of the equation, if a unique solution is to be obtained. Fewer independent 
boundary conditions than this will lead to a number of undetermined parameters 
in the solution, whilst an excess will usually mean that no acceptable solution is 
possible. 

For an nth-order equation the required n boundary conditions can take many 
forms, for example the value of y at n different values of x, or the value of any 
n—1 of the n derivatives dy/dx, d?y/dx’, ..., d"y/dx" together with that of y, all 
for the same value of x, or many intermediate combinations. 


14.2 First-degree first-order equations 


First-degree first-order ODEs contain only dy/dx equated to some function of x 
and y, and can be written in either of two equivalent standard forms, 


® = F(xy), Abs y)dx + BOs y)dy =O, 


where F(x, y) = —A(x, y)/B(x,y), and F(x,y), A(x, y) and B(x, y) are in general 
functions of both x and y. Which of the two above forms is the more useful 
for finding a solution depends on the type of equation being considered. There 
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are several different types of first-degree first-order ODEs that are of interest in 
the physical sciences. These equations and their respective solutions are discussed 
below. 


14.2.1 Separable-variable equations 


A separable-variable equation is one which may be written in the conventional 
form 


dy ; 
~ = fvaly), (14.3) 


where f(x) and g(y) are functions of x and y respectively, including cases in 
which f(x) or g(y) is simply a constant. Rearranging this equation so that the 
terms depending on x and on y appear on opposite sides (i.e. are separated), and 


integrating, we obtain 
dy i ; 
— = (x) dx. 
/ g(y) ‘ 


Finding the solution y(x) that satisfies (14.3) then depends only on the ease with 
which the integrals in the above equation can be evaluated. It is also worth 
noting that ODEs that at first sight do not appear to be of the form (14.3) can 
sometimes be made separable by an appropriate factorisation. 


Since the RHS of this equation can be factorised to give x(1 + y), the equation becomes 
separable and we obtain 


Now integrating both sides separately, we find 


x2 
In +y)=> +e, 


2 2 
i+y-e(F +e) = ae (5), 


where c and hence A is an arbitrary constant. < 


and so 


Solution method. Factorise the equation so that it becomes separable. Rearrange 
it so that the terms depending on x and those depending on y appear on opposite 
sides and then integrate directly. Remember the constant of integration, which can 
be evaluated if further information is given. 
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14.2.2 Exact equations 


An exact first-degree first-order ODE is one of the form 
F 0B 
A(x, y) dx + B(x, y)dy =0 — and for which ay = 3" (14.4) 
x 


In this case A(x,y)dx + B(x,y)dy is an exact differential, dU(x,y) say (see 
section 5.3). In other words 


oU oU 
Adx+Bdy=dU = ax dx 4 By dy, 
from which we obtain 
oU 
A(x,y) ==, (14.5) 
Ox 
oU 
B =. 14.6 
(x,y) By (14.6) 


Since 67U/@xdy = 6*U/édyéx we therefore require 


0A _ OB 
dy 6x’ 
If (14.7) holds then (14.4) can be written dU(x, y) = 0, which has the solution 
U(x, y) =c, where c is a constant and from (14.5) U(x, y) is given by 


(14.7) 


U(x, y) = [Aesnds + F(y). (14.8) 


The function F(y) can be found from (14.6) by differentiating (14.8) with respect 
to y and equating to B(x, y). 


> Solve 


se mer nee 
dx 


Rearranging into the form (14.4) we have 
(3x + y)dx+xdy =0, 


ie. A(x, y) = 3x+y and B(x, y) = x. Since 0A/dy = 1 = 0B/O0x, the equation is exact, and 
by (14.8) the solution is given by 


3) 2 
Ux = [Oxtydx+FO)=0 = E+ yx+ FO) =0. 


Differentiating U(x, y) with respect to y and equating it to B(x, y) = x we obtain dF /dy = 0, 
which integrates immediately to give F(y) = c2. Therefore, letting c = c, — cz, the solution 
to the original ODE is 

3x? 


Spe TS Gs 


472 


14.2 FIRST-DEGREE FIRST-ORDER EQUATIONS 


Solution method. Check that the equation is an exact differential using (14.7) then 
solve using (14.8). Find the function F(y) by differentiating (14.8) with respect to 
y and using (14.6). 


14.2.3 Inexact equations: integrating factors 


Equations that may be written in the form 


0A OB 
A(x, y)dx + B(x,y)dy =0 but for which 7 #4 —— (14.9) 
oy © Ox 


are known as inexact equations. However, the differential A dx + B dy can always 
be made exact by multiplying by an integrating factor p(x, y), which obeys 


(uA) _ O(uB) (14.10) 


oy Ox 


For an integrating factor that is a function of both x and y, Le. “ = u(x, y), there 
exists no general method for finding it; in such cases it may sometimes be found 
by inspection. If, however, an integrating factor exists that is a function of either 
x or y alone then (14.10) can be solved to find it. For example, if we assume 
that the integrating factor is a function of x alone, ie. uw = u(x), then (14.10) 
reads 


Rearranging this expression we find 


du 1/(0A OB 
Lu B 


ay _ =) dx = f(x) dx, 


where we require f(x) also to be a function of x only; indeed this provides a 
general method of determining whether the integrating factor w is a function of 
x alone. This integrating factor is then given by 


Mx) = exp{ [ soar} where f(x) = ; (= _ =) : (14.11) 


Similarly, if w = u(y) then 


1/0B a 
wiyy=exp{ f ewidy) where guy = 5 (2-4). cata) 
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Rearranging into the form (14.9), we have 


(4x + 3y?) dx + 2xy dy = 0, (14.13) 


ie. A(x, y) = 4x + 3y? and B(x, y) = 2xy. Now 
0A 0B 
eee eG pea) 
Vs ax Vs 


so the ODE is not exact in its present form. However, we see that 


1 (0A OB\ 2 

B\éy éx) x’ 
a function of x alone. Therefore an integrating factor exists that is also a function of x 
alone and, ignoring the arbitrary constant of integration, is given by 


uo) = exp {2 / <} = exp(2Inx) =» 


Multiplying (14.13) through by p(x) = x? we obtain 
(4x3 + 3x7y?) dx + 2x3 y dy = 4x3 dx + (3x7y’ dx + 2x3 y dy) = 0. 


oy 


By inspection this integrates immediately to give the solution x* + y?x*> = c, where c is a 
constant. <¢ 


Solution method. Examine whether f(x) and g(y) are functions of only x or y 
respectively. If so, then the required integrating factor is a function of either x or 
y only, and is given by (14.11) or (14.12) respectively. If the integrating factor is 
a function of both x and y, then sometimes it may be found by inspection or by 
trial and error. In any case, the integrating factor « must satisfy (14.10). Once the 
equation has been made exact, solve by the method of subsection 14.2.2. 


14.2.4 Linear equations 


Linear first-order ODEs are a special case of inexact ODEs (discussed in the 
previous subsection) and can be written in the conventional form 


i 
4+ P(x)y = O(x). (14.14) 


Such equations can be made exact by multiplying through by an appropriate 
integrating factor in a similar manner to that discussed above. In this case, 
however, the integrating factor is always a function of x alone and may be 
expressed in a particularly simple form. An integrating factor u(x) must be such 
that 
dy d 
BO) + HOP ODY = F [HOY] = HONOC), (14.15) 


1x 
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which may then be integrated directly to give 


wos = f wlsyQ(x) dx (14.16) 


The required integrating factor p(x) is determined by the first equality in (14.15), 
ie. 
dy du dy 


d 
an ey) ta ta uPy, 


which immediately gives the simple relation 


d 
- = p(x)P(x) =  p(x)=exp { [ Povas} : (14.17) 


The integrating factor is given immediately by 


L(x) = exp if 2x ax} =expx’. 


Multiplying through the ODE by p(x) = exp x? and integrating, we have 


yexpx? = 4 [ xexpx" dx = 2expx’ +e. 
The solution to the ODE is therefore given by y = 2 + cexp(—x’). < 


Solution method. Rearrange the equation into the form (14.14) and multiply by the 
integrating factor u(x) given by (14.17). The left- and right-hand sides can then be 
integrated directly, giving y from (14.16). 


14.2.5 Homogeneous equations 


Homogeneous equation are ODEs that may be written in the form 


dy _ A(x,y) =F (2) 


x 


aa (14.18) 


where A(x,y) and B(x,y) are homogeneous functions of the same degree. A 
function f(x, y) is homogeneous of degree n if, for any 2, it obeys 
F(Ax, Ay) = A" f(x, y). 


For example, if A = x*y — xy? and B = x3 + y? then we see that A and B are 
both homogeneous functions of degree 3. In general, for functions of the form of 
A and B, we see that for both to be homogeneous, and of the same degree, we 
require the sum of the powers in x and y in each term of A and B to be the same 
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(in this example equal to 3). The RHS of a homogeneous ODE can be written as 
a function of y/x. The equation may then be solved by making the substitution 
y = vx, so that 


This is now a separable equation and can be integrated directly to give 


Im a (14.19) 


Substituting y = vx we obtain 


dv 
v+x—=v-+tanv. 
dx 


Cancelling v on both sides, rearranging and integrating gives 


footed = fF =Ix-+e. 


[cotede = f SP av = in(sine) +e, 
ant 


But 


IN v 


so the solution to the ODE is y = xsin”' Ax, where A is a constant. < 


Solution method. Check to see whether the equation is homogeneous. If so, make 
the substitution y = vx, separate variables as in (14.19) and then integrate directly. 
Finally replace v by y/x to obtain the solution. 


14.2.6 Isobaric equations 
An isobaric ODE is a generalisation of the homogeneous ODE discussed in the 
previous section, and is of the form 


dy _ A(x,y) 


dx B(x,y)’ 


(14.20) 


where the equation is dimensionally consistent if y and dy are each given a weight 
m relative to x and dx, ie. if the substitution y = vx” makes it separable. 
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Rearranging we have 


(? + =) dx + 2yxdy =0. 
x 


Giving y and dy the weight m and x and dx the weight 1, the sums of the powers in each 
term on the LHS are 2m+ 1, 0 and 2m+ 1 respectively. These are equal if 2m+ 1 = 0, ie. 
if m = —4. Substituting y = vx" = vx”, with the result that dy = x7'/? dv — $vx~¥/? dx, 
we obtain 


Ay eee 
x 


which is separable and may be integrated directly to give $v? +Inx =c. Replacing v by 
y./X we obtain the solution 5y°x +Inx=c.< 


Solution method. Write the equation in the form Adx + Bdy = 0. Giving y and 
dy each a weight m and x and dx each a weight 1, write down the sum of powers 
in each term. Then, if a value of m that makes all these sums equal can be found, 
substitute y = vx" into the original equation to make it separable. Integrate the 
separated equation directly, and then replace v by yx" to obtain the solution. 


14.2.7 Bernoulli’s equation 


Bernoulli’s equation has the form 


e + P(x)y =Q(x)y” where n #0 or 1. (14.21) 


This equation is very similar in form to the linear equation (14.14), but is in fact 
non-linear due to the extra y" factor on the RHS. However, the equation can be 
made linear by substituting v = y!~" and correspondingly 


dy fy" dv 
dx \1—n/ dx’ 


Substituting this into (14.21) and dividing through by y”, we find 


# 1 POw = (1 —O(0), 
dx 


which is a linear equation and may be solved by the method described in 
subsection 14.2.4. 
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which is linear and may be solved by multiplying through by the integrating factor (see 


subsection 14.2.4) 


exp { 3 fF} exp(—3 In x) ~ 


This yields the solution 
v 
a 


—6x +c. 


Remembering that v = y~*, we obtain y~> = —6x* + cx?. 


Solution method. Rearrange the equation into the form (14.21) and make the sub- 
stitution v = y'!—". This leads to a linear equation in v, which can be solved by the 
method of subsection 14.2.4. Then replace v by y'~" to obtain the solution. 


14.2.8 Miscellaneous equations 


There are two further types of first-degree first-order equation that occur fairly 

regularly but do not fall into any of the above categories. They may be reduced 

to one of the above equations, however, by a suitable change of variable. 
Firstly, we consider 


Say een aay (14.22) 
dx 


where a, b and c are constants, i.e. x and y only appear on the RHS in the particular 
combination ax + by +c and not in any other combination or by themselves. This 
equation can be solved by making the substitution v = ax + by +c, in which case 


d dy 
Std os 
xX 


x ie =a t OF), (14.23) 


which is separable and may be integrated directly. 
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Making the substitution v = x + y + 1, we obtain, as in (14.23), 


dv 
dx 
which is separable and integrates directly to give 


A a = fax => tan'!v=x+e 
1402 7 v 


So the solution to the original ODE is tan!(x +y+1)=x+c1, where c; is a constant of 
integration. < 


=v +1, 


Solution method. In an equation such as (14.22), substitute v = ax+by+c to obtain 
a separable equation that can be integrated directly. Then replace v by ax + by +c 
to obtain the solution. 


Secondly, we discuss 


Gy eee Ne (14.24) 
dx ex+fy+g 
where a, b, c, e, f and g are all constants. This equation may be solved by letting 


x=X-+aand y=Y +f, where « and f are constants found from 


axz+bp+c=0 (14.25) 

ea+fB+g=0. (14.26) 
Then (14.24) can be written as 

dY _aX+bY 

dX eX+fY’ 


which is homogeneous and can be solved by the method of subsection 14.2.5. 
Note, however, that if a/e = b/f then (14.25) and (14.26) are not independent 
and so cannot be solved uniquely for « and £. However, in this case, (14.24) 
reduces to an equation of the form (14.22), which was discussed above. 


> Solve 


Let x = X +a and y= Y + Bf, where « and f obey the relations 
2a—58+3=0 
2a+ 48 —6 =0, 

which solve to give « = f = 1. Making these substitutions we find 
dY 2x-—5Y 
dx 2X+4Y’ 
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which is a homogeneous ODE and can be solved by substituting Y = vX (see subsec- 
tion 14.2.5) to obtain 


dv _ 2—Tv—4v? 
dX =X(2+4v) ° 


This equation is separable, and using partial fractions we find 


/ 2+4v dp > / dv = / dv dx 
2— Tv — 4v? 3/ 4v-1 3/ v+2 xX? 


which integrates to give 


InxX + + In(4v -—1)+ 3 In(v + 2) = c, 
or 
X3(4v — 1)(v + 2)? = exp 3c). 


Remembering that Y = vX, x = X + 1 and y= Y +1, the solution to the original ODE 
is given by (4y — x — 3)(y + 2x — 3)? = @, where cy. = exp 3c). < 


Solution method. If in (14.24) a/e # b/f then make the substitution x = X +4, 
y =Y-+8, where « and B are given by (14.25) and (14.26); the resulting equation 
is homogeneous and can be solved as in subsection 14.2.5. Substitute v = Y/X, 
X =x—aand Y =y—f to obtain the solution. If a/e = b/f then (14.24) is of 
the same form as (14.22) and may be solved accordingly. 


14.3 Higher-degree first-order equations 


First-order equations of degree higher than the first do not occur often in 
the description of physical systems, since squared and higher powers of first- 
order derivatives usually arise from resistive or driving mechanisms, when an 
acceleration or other higher-order derivative is also present. They do sometimes 
appear in connection with geometrical problems, however. 

Higher-degree first-order equations can be written as F(x, y,dy/dx) = 0. The 
most general standard form is 


p” + dy-1(x, y)p” | +--+ +ai(x, y)p + ao(x, y) = 0, (14.27) 


where for ease of notation we write p = dy/dx. If the equation can be solved for 
one of x, y or p then either an explicit or a parametric solution can sometimes be 
obtained. We discuss the main types of such equations below, including Clairaut’s 
equation, which is a special case of an equation explicitly soluble for y. 


14.3.1 Equations soluble for p 


Sometimes the LHS of (14.27) can be factorised into the form 
(p — Fi)(p — F)+++(p — Fn) = 0, (14.28) 
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where F; = F(x, y). We are then left with solving the n first-degree equations 
p = F;,(x, y). Writing the solutions to these first-degree equations as G;(x, y) = 0, 
the general solution to (14.28) is given by the product 


Gi (x, y)Ga(x, y) +++ Gulx, y) = 0. (14.29) 


(x3 + x7 + x + 1)p* — (3x? + 2x + 1)yp + 2xy? = 0. 


This equation may be factorised to give 
[(x + Dp — yO? + Dp — 2xy] = 0. 


Taking each bracket in turn we have 
dy 
x+1)——-y=0. 
CFD =o 
(7 + yy —2xy =0 
: dx P 


which have the solutions y — c(x + 1) = 0 and y —c(x? +1) = 0 respectively (see 
section 14.2 on first-degree first-order equations). Note that the arbitrary constants in 
these two solutions can be taken to be the same, since only one is required for a first-order 
equation. The general solution to (14.30) is then given by 


Ly —ex + I) [y—e(x? +1] =0. < 


Solution method. If the equation can be factorised into the form (14.28) then solve 
the first-order ODE p — F; = 0 for each factor and write the solution in the form 
G;(x,y) = 0. The solution to the original equation is then given by the product 
(14.29). 


14.3.2 Equations soluble for x 
Equations that can be solved for x, i.e. such that they may be written in the form 
x = F(y, p), (14.31) 


can be reduced to first-degree first-order equations in p by differentiating both 
sides with respect to y, so that 


dx 1 OF | 0Fdp 
dy p 6y Opdy 
This results in an equation of the form G(y, p) = 0, which can be used together 
with (14.31) to eliminate p and give the general solution. Note that often a singular 
solution to the equation will be found at the same time (see the introduction to 


this chapter). 
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6y?p? + 3xp—y =0. 


This equation can be solved for x explicitly to give 3x = (y/p)— 6yp. Differentiating both 
sides with respect to y, we find 


dx 3 1 ydp dp 
3 6y 12yp, 
dy pp pdy dy 
which factorises to give 
2 dp 
(1+ 6yp*) | 2p + =0. (14.33) 


Setting the factor containing dp/dy equal to zero gives a first-degree first-order equation 
in p, which may be solved to give py? = c. Substituting for p in (14.32) then yields the 
general solution of (14.32): 


y> = 3ex + 6c’. (14.34) 
If we now consider the first factor in (14.33), we find 6p?y = —1 as a possible solution. 
Substituting for p in (14.32) we find the singular solution 

8y3 + 3x? = 0. 


Note that the singular solution contains no arbitrary constants and cannot be found from 
the general solution (14.34) by any choice of the constant c. < 


Solution method. Write the equation in the form (14.31) and differentiate both 
sides with respect to y. Rearrange the resulting equation into the form G(y, p) = 0, 
which can be used together with the original ODE to eliminate p and so give the 
general solution. If G(y, p) can be factorised then the factor containing dp/dy should 
be used to eliminate p and give the general solution. Using the other factors in this 
fashion will instead lead to singular solutions. 


14.3.3 Equations soluble for y 


Equations that can be solved for y, i.e. are such that they may be written in the 
form 


y = F(x,p), (14.35) 
can be reduced to first-degree first-order equations in p by differentiating both 
sides with respect to x, so that 

dy OF oF dp 


dx? Ox" Opdx’ 


This results in an equation of the form G(x, p) = 0, which can be used together 
with (14.35) to eliminate p and give the general solution. An additional (singular) 
solution to the equation is also often found. 
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xp* + 2xp —y = 0. 


This equation can be solved for y explicitly to give y = xp? + 2xp. Differentiating both 
sides with respect to x, we find 


dp 
2 
at D; 


dp | 2 
p 2xpaZ tp + 2x4 


which after factorising gives 
dp 
+) (p+2x%8) =0. (14.37) 


To obtain the general solution of (14.36), we consider the factor containing dp/dx. This 
first-degree first-order equation in p has the solution xp” = c (see subsection 14.3.1), which 
we then use to eliminate p from (14.36). Thus we find that the general solution to (14.36) 
is 


(y —c)? = 4ex. (14.38) 
If instead, we set the other factor in (14.37) equal to zero, we obtain the very simple 
solution p = —1. Substituting this into (14.36) then gives 
xt+ty=0, 
which is a singular solution to (14.36). < 


Solution method. Write the equation in the form (14.35) and differentiate both 
sides with respect to x. Rearrange the resulting equation into the form G(x, p) = 0, 
which can be used together with the original ODE to eliminate p and so give the 
general solution. If G(x, p) can be factorised then the factor containing dp/dx should 
be used to eliminate p and give the general solution. Using the other factors in this 
fashion will instead lead to singular solutions. 


14.3.4 Clairaut’s equation 
Finally, we consider Clairaut’s equation, which has the form 
y = px + F(p) (14.39) 


and is therefore a special case of equations soluble for y, as in (14.35). It may be 
solved by a similar method to that given in subsection 14.3.3, but for Clairaut’s 
equation the form of the general solution is particularly simple. Differentiating 
(14.39) with respect to x, we find 


dy dp | dF dp dp {dF 
—=p= —+—— — | — = 0. 14.40 
i ge age. oo ae ae a0) 
Considering first the factor containing dp/dx, we find 
dp dy 
om Tee y C1X + C2. (14.41) 
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Since p = dy/dx = c1, if we substitute (14.41) into (14.39) we find cix +c) = 
c1x + F(ci). Therefore the constant cz is given by F(c:), and the general solution 
to (14.39) is 


y=c1x + F(c1), (14.42) 


Le. the general solution to Clairaut’s equation can be obtained by replacing p 
in the ODE by the arbitrary constant c;. Now, considering the second factor in 
(14.40), we also have 


a +x=0, (14.43) 
dp 
which has the form G(x, p) = 0. This relation may be used to eliminate p from 
(14.39) to give a singular solution. 


From (14.42) the general solution is y = cx + c?. But from (14.43) we also have 2p +x = 
0 > p = —x/2. Substituting this into (14.44) we find the singular solution x? +4y =0. < 


Solution method. Write the equation in the form (14.39), then the general solution 
is given by replacing p by some constant c, as shown in (14.42). Using the relation 
dF /dp+x =0 to eliminate p from the original equation yields the singular solution. 


14.4 Exercises 
14.1 A radioactive isotope decays in such a way that the number of atoms present at 
a given time, N(t), obeys the equation 
dN 
— =-IN. 
dt 


If there are initially No atoms present, find N(t) at later times. 
14.2 Solve the following equations by separation of the variables: 
(a) y’— xy? =0; 
(b) y’tan7!x — y1+x?)"! = 0; 
(c) x?y’+ xy? = 4y?. 
14.3 Show that the following equations either are exact or can be made exact, and 
solve them: 
(a) y(2x?y? + Dy! + x(y* + 1) = 0; 
(b) 2xy’+3x+y=0; 
(c) (cos?x + ysin2x)y’ + y? =0. 
14.4 Find the values of « and f that make 


dF(xy) = ( : 


a 
42 + =) dx + (xy? + 1) dy 


an exact differential. For these values solve F(x, y) = 0. 
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14.5 


14.6 


14.7 


14.8 


14.9 


14.10 


14.11 


14.12 


By finding suitable integrating factors, solve the following equations: 


(a) (1—x2)y’ + 2xy = (1 —x?)3/?; 
(b) y’— ycotx + cosec x = 0; 
(c) (x+y3)y’ = y (treat y as the independent variable). 


By finding an appropriate integrating factor, solve 


dy 2w+y?+x 
dx xy ; 


Find, in the form of an integral, the solution of the equation 


for a general function f(t). Find the specific solutions for 


(a) f(t) = HU), 
(b) f(t) = d(0), 
(c) f(t) = Ble" H(t) with B < a. 


For case (c), what happens if B > 0? 
A series electric circuit contains a resistance R, a capacitance C and a battery 
supplying a time-varying electromotive force V(t). The charge q on the capacitor 
therefore obeys the equation 
dq 4 
R—+=5=V(t). 
nttroTl 
Assuming that initially there is no charge on the capacitor, and given that 
V(t) = Vo sin at, find the charge on the capacitor as a function of time. 
Using tangential—polar coordinates (see exercise 2.20), consider a particle of mass 
m moving under the influence of a force f directed towards the origin O. By 
resolving forces along the instantaneous tangent and normal and making use of 
the result of exercise 2.20 for the instantaneous radius of curvature, prove that 
dv dr 
= —mv— and mv? = fp—. 
f dr tp dp 
Show further that h = mpv is a constant of the motion and that the law of force 
can be deduced from 


Use the result of exercise 14.9 to find the law of force, acting towards the origin, 
under which a particle must move so as to describe the following trajectories: 


(a) A circle of radius a that passes through the origin; 
(b) An equiangular spiral, which is defined by the property that the angle « 
between the tangent and the radius vector is constant along the curve. 


Solve 
dy 
(y-xn2 +2x+3y =0. 
dx 
A mass m is accelerated by a time-varying force «exp(—ft)v?, where v is its 
velocity. It also experiences a resistive force nv, where 7 is a constant, owing to 


its motion through the air. The equation of motion of the mass is therefore 


d m 
mo = aexp(—Bt)v? — nv. 
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14.13 


14.14 


14.15 


14.16 


14.17 


14.18 


14.19 


14.20 


Find an expression for the velocity v of the mass as a function of time, given 
that it has an initial velocity vo. 
Using the results about Laplace transforms given in chapter 13 for df/dt and 
tf(t), show, for a function y(t) that satisfies 

dy 

t—+(t-—l)y=0 * 

ae te Dy (*) 

with y(0) finite, that (s) = C(1 +s)~* for some constant C. 
Given that 


y(t) =t+ s a,t", 
n=2 


determine C and show that a, = (—1)""!/(n—1)!. Compare this result with that 
obtained by integrating (*) directly. 
Solve 
dy _ 1 
ae ay 
Solve 
dy _ x+y 


dx 3x+3y—4 
Ifu=1+tany, calculate d(Inu)/dy; hence find the general solution of 


dy 


nee tanxcos y (cosy + sin y). 


Solve 
d 
x(- 2x2y) 2 +y = 3x’y’, 
dx 


given that y(1) = 1/2. 

A reflecting mirror is made in the shape of the surface of revolution generated by 
revolving the curve y(x) about the x-axis. In order that light rays emitted from a 
point source at the origin are reflected back parallel to the x-axis, the curve y(x) 
must obey 


Seg 
where p = dy/dx. By solving this equation for x, find the curve y(x). 
Find the curve with the property that at each point on it the sum of the intercepts 
on the x- and y-axes of the tangent to the curve (taking account of sign) is equal 
to 1. 
Find a parametric solution of 


Vo 
x 


as follows. 


(a) Write an equation for y in terms of p = dy/dx and show that 


dp 

=pt(2 HN—. 

p=p' + (2px + 

(b) Using p as the independent variable, arrange this as a linear first-order 
equation for x. 
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14.21 


14.22 


14.23 


14.24 


14.25 


14.26 


14.27 


(c) Find an appropriate integrating factor to obtain 
_ Inp—pte 
(l—py ° 
which, together with the expression for y obtained in (a), gives a parameter- 
isation of the solution. 
(d) Reverse the roles of x and y in steps (a) to (c), putting dx/dy = p~', and 
show that essentially the same parameterisation is obtained. 


Using the substitutions uv = x? and v = y’, reduce the equation 


xy dy ‘ (x? +y? 1) 4. xy 0 
dx dx 


to Clairaut’s form. Hence show that the equation represents a family of conics 
and the four sides of a square. 
The action of the control mechanism on a particular system for an input f(t) is 
described, for t > 0, by the coupled first-order equations: 

p+4z=f(0), 

z-2z2=y+H4y. 
Use Laplace transforms to find the response y(t) of the system to a unit step 
input, f(t) = H(t), given that y(0) = 1 and z(0) = 0. 


Questions 23 to 31 are intended to give the reader practice in choosing an approp- 
riate method. The level of difficulty varies within the set ; if necessary, the hints may 
be consulted for an indication of the most appropriate approach. 


Find the general solutions of the following: 


dy xy 2 
a) —+——S =x; (b =—-y’. 
@) P+ aee 0) 
Solve the following first-order equations for the boundary conditions given: 
(a) —(y/x)=1, yd) =—1; 
(b) y’—ytanx=1, — y(n /4) =3; 
(c) yyw /xr = 1/4, y= 1; 
(d) y’—y?/x? = 1/4, y(1) = 1/2. 
An electronic system has two inputs, to each of which a constant unit signal is 
applied, but starting at different times. The equations governing the system thus 
take the form 
x+2y = H(t), 
y—2x = H(t — 3). 
Initially (at t= 0), x = 1 and y = 0; find x(t) at later times. 
Solve the differential equation 


d 
sin x2 +2ycosx = 1, 
dx 


subject to the boundary condition y(z/2) = 1. 
Find the complete solution of 


dy\* _ydy A 
ao —-i—-4+ —=0, 
(4) eax? % , 
where A is a positive constant. 
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14.28 


14.29 


14.30 


14.31 


14.1 
14.3 


14.5 


14.7 


14.9 


14.11 


14.13 


14.15 
14.17 


14.19 


14.21 


14.23 


14.25 


14.27 


14.29 


Find the solution of 
dy 
(Sx +y- DS =3(x+yt1). 


Find the solution y = y(x) of 


dy y 
xa tY— sap = 9 
subject to y(1) = 1. 
Find the solution of 
dy 
(2sin y — ye =tany, 


if (a) y(O) = 0, and (b) y(0) = 2/2. 
Find the family of solutions of 


Wy dy\> d 
$+(2) +2 0 


dx? dx dx 
that satisfy y(0) = 0. 


14.5 Hints and answers 


N(t) = No exp(—At). 

(a) exact, x*y4 +x? + y? = c; (b) IF = x77, x'?(x + y) = ¢; (c) IF = 
sec? x, y>tanx+y=c. 

(a) IF = (1—x?)?, y = (1 —x*)(k + sin! x); (b) IF = cosec x, leading to 
y = ksinx + cosx; (c) exact equation is y~!(dx/dy) — xy~? = y, leading to 
x = y(k + y?/2). 

yt) = et flo tel /*f(t)dt’; (a) y(t) = 1 — e/*; (b) y(t) = te; (0) y(t) = 
(e/* — e/B) /(, — B). It becomes case (b). 

Note that, if the angle between the tangent and the radius vector is «, then 
cosa = dr/ds and sing = p/r. 

Homogeneous equation, put y = vx to obtain (1 — v)(v? + 2v + 2)-! dv = x“! dx; 
write 1—v as 2—(1+v), and > +2v+2as1+(1+v); 

A[x? + (x + y)?] = exp {4 tan! [(x + y)/x]}. 

(1+ s)(dy/ds) + 29 = 0. C = 1; use separation of variables to show directly that 


y(t) = te“. 
The equation is of the form of (14.22), setv =x+y;x+3y+2In(x+y—2) =A. 
The equation is isobaric with weight y = —2; setting y = vx? gives 


vo (1—v) "(1 — 2v) dv = x7! dx; 4xy(1 — x?y) = 1. 

The curve must satisfy y = (1—p~!)-!(1—x+px), which has solution x = (p—1)~?, 
leading to y= (1+ Jxy orx=(1+ afyes the singular solution p’ = 0 gives 
straight lines joining (0,0) and (0,1 — 9) for any 0. 

v = qu+q/(q—1), where q = dv/du. General solution y* = cx? + c/(c — 1), 
hyperbolae for c > 0 and ellipses for c < 0. Singular solution y = +(x + 1). 

(a) Integrating factor is (a?7+x7)!/?, y = (a? +x?)/3+A(a@+x’)~'/?; (b) separable, 
y=x(x?+Ax +4). 

Use Laplace transforms; xs(s? + 4) = s + s* — 2e 
x(t) = 5 sin 2t + cos 2t — 5H(t —3)+ 5 cos(2t — 6)H(t — 3). 

This is Clairaut’s equation with F(p) = A/p. General solution y = cx + A/c; 
singular solution, y = 2,/Ax. 

Either Bernoulli's equation with n = 2 or an isobaric equation with m = 3/2; 
y(x) = 5x3/7/(2 + 3x*/?), 


—3s. 
> 
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14.31 Show that p = (Ce* — 1)~', where p = dy/dx; y = In[C — e~*)/(C — 1)] or 
In[D — (D — 1)e~*] or In(e~“K +1—e7*) + K. 
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15 


Higher-order ordinary differential 
equations 


Following on from the discussion of first-order ordinary differential equations 
(ODEs) given in the previous chapter, we now examine equations of second and 
higher order. Since a brief outline of the general properties of ODEs and their 
solutions was given at the beginning of the previous chapter, we will not repeat 
it here. Instead, we will begin with a discussion of various types of higher-order 
equation. This chapter is divided into three main parts. We first discuss linear 
equations with constant coefficients and then investigate linear equations with 
variable coefficients. Finally, we discuss a few methods that may be of use in 
solving general linear or non-linear ODEs. Let us start by considering some 
general points relating to all linear ODEs. 

Linear equations are of paramount importance in the description of physical 
processes. Moreover, it is an empirical fact that, when put into mathematical 
form, many natural processes appear as higher-order linear ODEs, most often 
as second-order equations. Although we could restrict our attention to these 
second-order equations, the generalisation to nth-order equations requires little 
extra work, and so we will consider this more general case. 

A linear ODE of general order n has the form 


in 


d?- 1 y 


t An-1 (x) dx 


an() beta tagcoy = fo). (15) 


dx" 
If f(x) = 0 then the equation is called homogeneous; otherwise it is inhomogeneous. 
The first-order linear equation studied in subsection 14.2.4 is a special case of 
(15.1). As discussed at the beginning of the previous chapter, the general solution 
to (15.1) will contain n arbitrary constants, which may be determined if n boundary 
conditions are also provided. 

In order to solve any equation of the form (15.1), we must first find the 
general solution of the complementary equation, i.e. the equation formed by setting 
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f(x) =0: 
mn m—1 


d d 
ay(x)— + Gn—1(X) us 


dx" dxt 
To determine the general solution of (15.2), we must find n linearly independent 
functions that satisfy it. Once we have found these solutions, the general solution 
is given by a linear superposition of these n functions. In other words, if the n 
solutions of (15.2) are y;(x), yo(x),..., n(x), then the general solution is given by 
the linear superposition 


d 
$+ + a(x + ao(x)y = 0. (15.2) 


Ye(X) = cryi(x) + €2y2(x) +++ + CnYn(X), (15.3) 


where the c,, are arbitrary constants that may be determined if n boundary 
conditions are provided. The linear combination y,(x) is called the complementary 
function of (15.1). 

The question naturally arises how we establish that any n individual solutions to 
(15.2) are indeed linearly independent. For n functions to be linearly independent 


over an interval, there must not exist any set of constants c1, C2,...,C, such that 
cC1yi(X) + coya(x) + +++ + CnYn(x) = 0 (15.4) 
over the interval in question, except for the trivial case cy = c7 =--- =c, = 0. 


A statement equivalent to (15.4), which is perhaps more useful for the practical 
determination of linear independence, can be found by repeatedly differentiating 
(15.4), n— 1 times in all, to obtain n simultaneous equations for cj, c2,...,Cn: 


C1 V(X) + Cry2(X) + +++ + CnYa(x) = 0 


c1y1'(x) + cCoyo'(X) +++ + enn! (Xx) = 0 
(15.5) 


cry(x) 4 ey? is th aie cy) (x) = 0, 


where the primes denote differentiation with respect to x. Referring to the 
discussion of simultaneous linear equations given in chapter 8, if the determinant 
of the coefficients of c;, C2,...,C, 1s non-zero then the only solution to equations 
(15.5) is the trivial solution cy) = c) =--- =c, = 0. In other words, the n functions 
y1(x), y2(X),.-., n(x) are linearly independent over an interval if 


V1 y2 see Yn 


+0 (15.6) 


Wyn. =| 


1 . ee 
ye A A Ge? 


over that interval; W(y1, y2,.--, Yn) is called the Wronskian of the set of functions. 
It should be noted, however, that the vanishing of the Wronskian does not 
guarantee that the functions are linearly dependent. 
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If the original equation (15.1) has f(x) = 0 (i.e. it is homogeneous) then of 
course the complementary function y,(x) in (15.3) is already the general solution. 
If, however, the equation has f(x) ¥ 0 (i.e. it is inhomogeneous) then y,(x) is only 
one part of the solution. The general solution of (15.1) is then given by 


Y(X) = Ye(X) + Yp(), (15.7) 


where yp(x) is the particular integral, which can be any function that satisfies (15.1) 
directly, provided it is linearly independent of y,(x). It should be emphasised for 
practical purposes that any such function, no matter how simple (or complicated), 
is equally valid in forming the general solution (15.7). 

It is important to realise that the above method for finding the general solution 
to an ODE by superposing particular solutions assumes crucially that the ODE 
is linear. For non-linear equations, discussed in section 15.3, this method cannot 
be used, and indeed it is often impossible to find closed-form solutions to such 
equations. 


15.1 Linear equations with constant coefficients 


If the a,, in (15.1) are constants rather than functions of x then we have 


1 
Bie ieee gaye fOe. (15.8) 
dx 


Equations of this sort are very common throughout the physical sciences and 
engineering, and the method for their solution falls into two parts as discussed 
in the previous section, 1.e. finding the complementary function y,(x) and finding 
the particular integral y,(x). If f(x) = 0 in (15.8) then we do not have to find 
a particular integral, and the complementary function is by itself the general 
solution. 


15.1.1 Finding the complementary function y,(x) 


The complementary function must satisfy 


d 
et pate ae +aoy =0 (15.9) 


and contain n arbitrary constants (see equation (15.3)). The standard method 
for finding y,(x) is to try a solution of the form y = Ae**, substituting this into 
(15.9). After dividing the resulting equation through by Ae**, we are left with a 
polynomial equation in 4 of order n; this is the auxiliary equation and reads 


And” + yA") +++» bapa tay =0. (15.10) 
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In general the auxiliary equation has n roots, say (1, /2,...,4n. In certain cases, 
some of these roots may be repeated and some may be complex. The three main 
cases are as follows. 


(i) 


(il) 


(iii) 


All roots real and distinct. In this case the n solutions to (15.9) are exp Amx 
for m = 1 to n. It is easily shown by calculating the Wronskian (15.6) 
of these functions that if all the 4, are distinct then these solutions are 
linearly independent. We can therefore linearly superpose them, as in 
(15.3), to form the complementary function 


Ve(x) = eye" + cpe?* $+ + cye?"™®, (15.11) 


Some roots complex. For the special (but usual) case that all the coefficients 
dm in (15.9) are real, if one of the roots of the auxiliary equation (15.10) 
is complex, say «+ if, then its complex conjugate « — if is also a root. In 
this case we can write 


cpe FBX 4. eye iB)X — eo (di cos Bx + dy sin Bx) 


= ae { sin oxo (15.12) 


cos 


where A and ¢ are arbitrary constants. 

Some roots repeated. If, for example, 24; occurs k times (k > 1) as a root 
of the auxiliary equation, then we have not found n linearly independent 
solutions of (15.9); formally the Wronskian (15.6) of these solutions, having 
two or more identical columns, is equal to zero. We must therefore find 
k—1 further solutions that are linearly independent of those already found 
and also of each other. By direct substitution into (15.9) we find that 


xe4®, et, Ay xk-l pix 


are also solutions, and by calculating the Wronskian it is easily shown that 
they, together with the solutions already found, form a linearly independent 
set of n functions. Therefore the complementary function is given by 


Ve(x) = (ey + eax Foe Hegx! Me™ $ cy re™ + cp pge™® +--+ cn”. 
(15.13) 
If more than one root is repeated the above argument is easily extended. 
For example, suppose as before that 4; is a k-fold root of the auxiliary 
equation and, further, that 22 is an /-fold root (of course, k > 1 and! > 1). 
Then, from the above argument, the complementary function reads 


Vo(x) = (cy +oox tee e+ px e4tx 
+ (char + cegox tee Fegxh Ye 


+ Cee $F cy p gery foes bene. (15.14) 


22x 
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> Find the complementary function of the equation 


Setting the RHS to zero, substituting y = Ae* and dividing through by Ae** we obtain 
the auxiliary equation 

V-—2A+1=0. 
The root 2 = 1 occurs twice and so, although e* is a solution to (15.15), we must find 
a further solution to the equation that is linearly independent of e*. From the above 
discussion, we deduce that xe* is such a solution, so that the full complementary function 
is given by the linear superposition 


Vo(X) = (C1 + px)e*. 


Solution method. Set the RHS of the ODE to zero (if it is not already so), and 
substitute y = Ae. After dividing through the resulting equation by Ae, obtain 
an nth-order polynomial equation in 2 (the auxiliary equation, see (15.10)). Solve 
the auxiliary equation to find the n roots, 41, 42,...,4n, say. If all these roots are 
real and distinct then y,(x) is given by (15.11). If, however, some of the roots are 
complex or repeated then y,(x) is given by (15.12) or (15.13), or the extension 
(15.14) of the latter, respectively. 


15.1.2 Finding the particular integral y)(x) 


There is no generally applicable method for finding the particular integral y,(x) 
but, for linear ODEs with constant coefficients and a simple RHS, p(x) can often 
be found by inspection or by assuming a parameterised form similar to f(x). The 
latter method is sometimes called the method of undetermined coefficients. If f(x) 
contains only polynomial, exponential, or sine and cosine terms then, by assuming 
a trial function for yp(x) of similar form but one which contains a number of 
undetermined parameters and substituting this trial function into (15.9), the 
parameters can be found and yp(x) deduced. Standard trial functions are as 
follows. 


(i) If f(x) = ae™ then try 
Yp(X) = be™. 
(ii) If f(x) = a; sinrx + a) cosrx (a; or a) may be zero) then try 
Yp(X) = by sinrx + bo cos rx. 
(iii) If f(x) = a9 + ayx+---+ayx% (some a may be zero) then try 


Yp(x) = bo + dix +--+ + byx%, 
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(iv) If f(x) is the sum or product of any of the above then try yp(x) as the 
sum or product of the corresponding individual trial functions. 


It should be noted that this method fails if any term in the assumed trial 
function is also contained within the complementary function y,(x). In such a 
case the trial function should be multiplied by the smallest integer power of x 
such that it will then contain no term that already appears in the complementary 
function. The undetermined coefficients in the trial function can now be found 
by substitution into (15.8). 

Three further methods that are useful in finding the particular integral y,(x) are 
those based on Green’s functions, the variation of parameters, and a change in the 
dependent variable using knowledge of the complementary function. However, 
since these methods are also applicable to equations with variable coefficients, a 
discussion of them is postponed until section 15.2. 


> Find a particular integral of the equation 


From the above discussion our first guess at a trial particular integral would be y,(x) = be*. 
However, since the complementary function of this equation is y,(x) = (c; + cox)e* (as 
in the previous subsection), we see that e* is already contained in it, as indeed is xe*. 
Multiplying our first guess by the lowest integer power of x such that the result does not 
appear in y.(x), we therefore try yp(x) = bx’e*. Substituting this into the ODE, we find 
that b = 1/2, so the particular integral is given by yp(x) = x*e*/2. 


Solution method. If the RHS of an ODE contains only functions mentioned at the 
start of this subsection then the appropriate trial function should be substituted 
into it, thereby fixing the undetermined parameters. If, however, the RHS of the 
equation is not of this form then one of the more general methods outlined in sub- 
sections 15.2.3—15.2.5 should be used; perhaps the most straightforward of these is 
the variation-of-parameters method. 


15.1.3 Constructing the general solution y.(x) + Yp(x) 


As stated earlier, the full solution to the ODE (15.8) is found by adding together 
the complementary function and any particular integral. In order to illustrate 
further the material discussed in the last two subsections, let us find the general 
solution to a new example, starting from the beginning. 
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@y 


ae +4y = x’ sin 2x. 


First we set the RHS to zero and assume the trial solution y = Ae**. Substituting this into 
(15.16) leads to the auxiliary equation 


24+4=0 = A=H2i. (15.17) 
Therefore the complementary function is given by 
Ve(X) = ce* + oe? = d; cos 2x + do sin 2x. (15.18) 


We must now turn our attention to the particular integral y,(x). Consulting the list of 
standard trial functions in the previous subsection, we find that a first guess at a suitable 
trial function for this case should be 


(ax? + bx + c) sin2x + (dx? + ex + f) cos 2x. (15.19) 


However, we see that this trial function contains terms in sin2x and cos 2x, both of which 
already appear in the complementary function (15.18). We must therefore multiply (15.19) 
by the smallest integer power of x which ensures that none of the resulting terms appears 
in y,(x). Since multiplying by x will suffice, we finally assume the trial function 


(ax? + bx? + ex) sin 2x + (dx? + ex” + fx) cos 2x. (15.20) 


Substituting this into (15.16) to fix the constants appearing in (15.20), we find the particular 
integral to be 


3 2 


; x ales _ x 
Yp(x) D cos 2x + 16 sin 2x 4 5) cos 2x. (15.21) 


The general solution to (15.16) then reads 


y(x) — VelX) Bi Yp(x) 
3 2 


: x Rm: x 
d, cos 2x + d> sin 2x Tq 82x + 76 2x + 37 008 2%. < 


15.1.4 Linear recurrence relations 


Before continuing our discussion of higher-order ODEs, we take this opportunity 
to introduce the discrete analogues of differential equations, which are called 
recurrence relations (or sometimes difference equations). Whereas a differential 
equation gives a prescription, in terms of current values, for the new value of a 
dependent variable at a point only infinitesimally far away, a recurrence relation 
describes how the next in a sequence of values u,, defined only at (non-negative) 
integer values of the ‘independent variable’ n, is to be calculated. 

In its most general form a recurrence relation expresses the way in which u,41 
is to be calculated from all the preceding values uo, u1,... ,u,. Just as the most 
general differential equations are intractable, so are the most general recurrence 
relations, and we will limit ourselves to analogues of the types of differential 
equations studied earlier in this chapter, namely those that are linear, have 
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constant coefficients and possess simple functions on the RHS. Such equations 
occur over a broad range of engineering and statistical physics as well as in the 
realms of finance, business planning and gambling! They form the basis of many 
numerical methods, particularly those concerned with the numerical solution of 
ordinary and partial differential equations. 

A general recurrence relation is exemplified by the formula 


N-1 
Unt = > On +k, (15.22) 
r=0 


where N and the a, are fixed and k is a constant or a simple function of n. 
Such an equation, involving terms of the series whose indices differ by up to N 
(ranging from n—N +1 to n), is called an Nth-order recurrence relation. It is clear 
that, given values for uo, u1,... ,un—1, this is a definitive scheme for generating the 
series and therefore has a unique solution. 

Parallelling the nomenclature of differential equations, if the term not involving 
any u, is absent, i.e. k = 0, then the recurrence relation is called homogeneous. 
The parallel continues with the form of the general solution of (15.22). If v, is 
the general solution of the homogeneous relation, and w, is any solution of the 
full relation, then 


Un = Un + Wn 


is the most general solution of the complete recurrence relation. This is straight- 
forwardly verified as follows: 


Unt = Und. + Wott 


N-1 N-1 
= ) AyVp—y + ) ayWr-r +k 
r=0 r=0 


N-1 

De Ay (Up—r + War) +k 

r=0 
1 


ll 


Of course, if k = 0 then w, = 0 for all n is a trivial particular solution and the 
complementary solution, v,, is itself the most general solution. 


First-order recurrence relations 
First-order relations, for which N = 1, are exemplified by 
Unt1 = au, +k, (15.23) 
with uo specified. The solution to the homogeneous relation is immediate, 


Un = Ca", 
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and, if k is a constant, the particular solution is equally straightforward: w, = K 
for all n, provided K is chosen to satisfy 


K =aK +k, 


ie. K =k(1—a)~!. This will be sufficient unless a = 1, in which case u, = uo +nk 
is obvious by inspection. 
Thus the general solution of (15.23) is 


= oe —a) a#1, 


= 15.24 
" Ug + nk a=1. ( ) 


If uo is specified for the case of a 1 then C must be chosen as C = up—k/(1—a), 
resulting in the equivalent form 

1-a" 
l-a 


We now illustrate this method with a worked example. 


Un = ua" +k (15.25) 


>A house-buyer borrows capital B from a bank that charges a fixed annual rate of interest 
R%. If the loan is to be repaid over Y years, at what value should the fixed annual payments 


P, made at the end of each year, be set? For a loan over 25 years at 6%, what percentage 
of the first year’s payment goes towards paying off the capital? 


Let u, denote the outstanding debt at the end of year n, and write R/100 = r. Then the 
relevant recurrence relation is 


Uns = u,(1 oh r) —P 
with uo = B. From (15.25) we have 


1—(1+r)" 
n= BL " — P——____, 
ie a PETE 
As the loan is to be repaid over Y years, uy = 0 and thus 
poe 
(l+r)¥ -1 


The first year’s interest is rB and so the fraction of the first year’s payment going 
towards capital repayment is (P —rB)/P, which, using the above expression for P, is equal 
to (1+1r)~’. With the given figures, this is (only) 23%. < 


With only small modifications, the method just described can be adapted to 
handle recurrence relations in which the constant k in (15.23) is replaced by ka”, 
ie. the relation is 


Unt = AU, + ka". (15.26) 


As for an inhomogeneous linear differential equation (see subsection 15.1.2), we 
may try as a potential particular solution a form which resembles the term that 
makes the equation inhomogeneous. Here, the presence of the term ka" indicates 
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that a particular solution of the form u, = Aa" should be tried. Substituting this 
into (15.26) gives 


Agit! = aAg" + ko", 


from which it follows that A = k/(«— a) and that there is a particular solution 
having the form u, = ka"/(« — a), provided « # a. For the special case « = a, the 
reader can readily verify that a particular solution of the form u, = Ana" is appro- 
priate. This mirrors the corresponding situation for linear differential equations 
when the RHS of the differential equation is contained in the complementary 
function of its LHS. 

In summary, the general solution to (15.26) is 


fe t+ka"/(a—a) a#a, 
Un = 


15.27 
Coa" + kno"! a=a, ( ) 


with Cy; = uy —k/(a —a) and Cy = uo. 


Second-order recurrence relations 


We consider next recurrence relations that involve u,_; in the prescription for 
Un41 and treat the general case in which the intervening term, u,,, is also present. 
A typical equation is thus 


Untt = Auy + buy_1 +k. (15.28) 
As previously, the general solution of this is u, =v, + Wp», where v, satisfies 
Unt1 = Ay + bey» (15.29) 


and w, is any particular solution of (15.28); the proof follows the same lines as 
that given earlier. 

We have already seen for a first-order recurrence relation that the solution to 
the homogeneous equation is given by terms forming a geometric series, and we 
consider a corresponding series of powers in the present case. Setting v, = AA” in 
(15.29) for some 4, as yet undetermined, gives the requirement that 2 should satisfy 


Ad"! = ada" + DAI, 


Dividing through by A4”~! (assumed non-zero) shows that 4 could be either of 
the roots, 2; and J, of 


i? —ah—b =0, (15.30) 


which is known as the characteristic equation of the recurrence relation. 

That there are two possible series of terms of the form A” is consistent with the 
fact that two initial values (boundary conditions) have to be provided before the 
series can be calculated by repeated use of (15.28). These two values are sufficient 
to determine the appropriate coefficient A for each of the series. Since (15.29) is 
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both linear and homogeneous, and is satisfied by both v, = AZ} and v, = BA%, its 
general solution is 


Un = AA + BA. 


If the coefficients a and b are such that (15.30) has two equal roots, ie. a* = —4b, 
then, as in the analogous case of repeated roots for differential equations (see 
subsection 15.1.1(ili)), the second term of the general solution is replaced by Bn/} 
to give 


Vn = (A+ Bn)/j. 


Finding a particular solution is straightforward if k is a constant: a trivial but 
adequate solution is w, = k(1— a —b)~! for all n. As with first-order equations, 
particular solutions can be found for other simple forms of k by trying functions 
similar to k itself. Thus particular solutions for the cases k = Cn and k = Da" 
can be found by trying w, = E + Fn and w, = Ga” respectively. 


> Find the value of uy6 if the series u, satisfies 


Untt + 4Uy + 3Un—1 =n 
for n= 1, with up =1 and uy, = —1. 


We first solve the characteristic equation, 
V+41.4+3=0, 
to obtain the roots 2 = —1 and 4 = —3. Thus the complementary function is 
vy, = A(—1)" + B(—3)”. 
In view of the form of the RHS of the original relation, we try 
w,=E+Fn 
as a particular solution and obtain 
E+F(n+1)+4(E + Fn) + 3[E + F(n—1)] =n, 
yielding F = 1/8 and E = 1/32. 


Thus the complete general solution is 


n n 4 n 
Un = A(—1)" + B(—3)" + 3 + 35> 


and now using the given values for up and u; determines A as 7/8 and B as 3/32. Thus 


5 [28(—1)" + 3(—3)" +.4n + 1]. 


Un 


Finally, substituting n = 16 gives uijg = 4035 633, a value the reader may (or may not) 
wish to verify by repeated application of the initial recurrence relation. < 
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Higher-order recurrence relations 


It will be apparent that linear recurrence relations of order N > 2 do not present 
any additional difficulty in principle, though two obvious practical difficulties are 
(i) that the characteristic equation is of order N and in general will not have roots 
that can be written in closed form and (ii) that a correspondingly large number 
of given values is required to determine the N otherwise arbitrary constants in 
the solution. The algebraic labour needed to solve the set of simultaneous linear 
equations that determines them increases rapidly with N. We do not give specific 
examples here, but some are included in the exercises at the end of the chapter. 


15.1.5 Laplace transform method 


Having briefly discussed recurrence relations, we now return to the main topic 
of this chapter, i.e. methods for obtaining solutions to higher-order ODEs. One 
such method is that of Laplace transforms, which is very useful for solving 
linear ODEs with constant coefficients. Taking the Laplace transform of such an 
equation transforms it into a purely algebraic equation in terms of the Laplace 
transform of the required solution. Once the algebraic equation has been solved 
for this Laplace transform, the general solution to the original ODE can be 
obtained by performing an inverse Laplace transform. One advantage of this 
method is that, for given boundary conditions, it provides the solution in just 
one step, instead of having to find the complementary function and particular 
integral separately. 

In order to apply the method we need only two results from Laplace transform 
theory (see section 13.2). First, the Laplace transform of a function f(x) is defined 
by 


f(s) = : * ese f(x) dx, (15.31) 


from which we can derive the second useful relation. This concerns the Laplace 
transform of the nth derivative of f(x): 


FON(s) = s*F(s) —s™'f(0) —s"°f'(0) —--- — sf") — f°"), 
(15.32) 


where the primes and superscripts in parentheses denote differentiation with 
respect to x. Using these relations, along with table 13.1, on p. 455, which gives 
Laplace transforms of standard functions, we are in a position to solve a linear 
ODE with constant coefficients by this method. 


501 


HIGHER-ORDER ORDINARY DIFFERENTIAL EQUATIONS 


subject to the boundary conditions y(0) = 2, y'(0) = 1. 


Taking the Laplace transform of (15.33) and using the table of standard results we obtain 


a 7 7 2 
s°p(s) — sy(0) — y'(0) — 3 [sp(s) — y(0)] + 25(s) = SH’ 
which reduces to 
2 
2 _— y _— =e 
(s* — 3s + 2)p(s) — 2s +5 aT (15.34) 


Solving this algebraic equation for f(s), the Laplace transform of the required solution to 
(15.33), we obtain 
_ 2s? — 3s —3 1 2 1 
y(s) f , 
(st1)(s—1)(s—2) 3(s+1) s—1  3(s—2) 
where in the final step we have used partial fractions. Taking the inverse Laplace transform 
of (15.35), again using table 13.1, we find the specific solution to (15.33) to be 


(15.35) 


y(x) = fe + 2e* — te. 


Note that if the boundary conditions in a problem are given as symbols, rather 
than just numbers, then the step involving partial fractions can often involve 
a considerable amount of algebra. The Laplace transform method is also very 
convenient for solving sets of simultaneous linear ODEs with constant coefficients. 


> Two electrical circuits, both of negligible resistance, each consist of a coil having self- 
inductance L and a capacitor having capacitance C. The mutual inductance of the two 
circuits is M. There is no source of e.m,f. in either circuit. Initially the second capacitor 
is given a charge CVo, the first capacitor being uncharged, and at time t = 0 a switch in 
the second circuit is closed to complete the circuit. Find the subsequent current in the first 
circuit. 


Subject to the initial conditions q\(0) = q1(0) = q2(0) = 0 and qo(0) = CVo = Vo/G, say, 
we have to solve 
Li, + Ma + Gai = 9, 
Mqi + La + Gq = 0. 
On taking the Laplace transform of the above equations, we obtain 
(Ls? + G)qi + Ms°Go = sMVoC, 
Ms’q; + (Ls? + G)q2 = sLVoC. 


Eliminating g. and rewriting as an equation for q, we find 
a MVos 
au(s) 2 2 
[(L + M)s? + G][(L — M)s? + G] 
Vo (L + M)s (L— M)s 
2G |(L+M)s+G (L—M)s?+G 
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Using table 13.1, 
1 


q(t) = 3VoC(cos wt — cos wat), 
where w7(L + M) = G and w3(L— M) =G. Thus the current is given by 
i(t) = $VoC(or sin wot — w1 sin@t). < 

Solution method. Perform a Laplace transform, as defined in (15.31), on the entire 
equation, using (15.32) to calculate the transform of the derivatives. Then solve the 
resulting algebraic equation for y(s), the Laplace transform of the required solution 
to the ODE. By using the method of partial fractions and consulting a table of 
Laplace transforms of standard functions, calculate the inverse Laplace transform. 
The resulting function y(x) is the solution of the ODE that obeys the given boundary 
conditions. 


15.2 Linear equations with variable coefficients 


There is no generally applicable method of solving equations with coefficients 
that are functions of x. Nevertheless, there are certain cases in which a solution is 
possible. Some of the methods discussed in this section are also useful in finding 
the general solution or particular integral for equations with constant coefficients 
that have proved impenetrable by the techniques discussed above. 


15.2.1 The Legendre and Euler linear equations 


Legendre’s linear equation has the form 


nay dy : 
an(ax + B)"—= +--+ + ar(ax + B)— + aoy = f(x), (15.36) 
dx" dx 
where «, f and the a, are constants and may be solved by making the substitution 
ax + B = e'. We then have 
dy dtdy «a dy 


dx dxdt ax + B dt 
Py ddy _ ot? (2 *) 


dx? dxdx  (ax+PP \de at 


and so on for higher derivatives. Therefore we can write the terms of (15.36) as 


dy __dy 
(ax +B) = a7, 
My _ yd (4a _ 
(ax + BY a5 =e a ys 


(15.37) 
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Substituting equations (15.37) into the original equation (15.36), the latter becomes 
a linear ODE with constant coefficients, i.e. 


nad(d d dy _,(e-B 
aa (Lat) (Sant t)y te bat tay =F ( z > 


which can be solved by the methods of section 15.1. 
A special case of Legendre’s linear equation, for which « = 1 and f = 0, is 
Euler’s equation, 


pies she crc ieee deny = f(x); (15.38) 

dx" dx 
it may be solved in a similar manner to the above by substituting x = e’. If 
f(x) = 0 in (15.38) then substituting y = x* leads to a simple algebraic equation 
in 2, which can be solved to yield the solution to (15.38). In the event that the 
algebraic equation for 4 has repeated roots, extra care is needed. If 2, is a k-fold 
root (k > 1) then the k linearly independent solutions corresponding to this root 


are x", x4 Inx,...,x7"(In x)k!, 


> Solve 


a d 
eet +x —4y =0 (15.39) 


by both of the methods discussed above. 


First we make the substitution x = e', which, after cancelling e', gives an equation with 
constant coefficients, i.e. 


dt \dt dt de 


Using the methods of section 15.1, the general solution of (15.40), and therefore of (15.39), 
is given by 


) 2 
= (+ ‘yee 4y =0 Pa ay 2G (15.40) 


2t 2 


y =cye* + oe = x? +x. 


Since the RHS of (15.39) is zero, we can reach the same solution by substituting y = x? 
into (15.39). This gives 
MA — 1)x* + Ax* — 4x? = 0, 
which reduces to 
(7 —4)x’ = 0. 
This has the solutions 2 = +2, so we obtain again the general solution 


2? 
y= Cx? +cox ~. 4 


Solution method. If the ODE is of the Legendre form (15.36) then substitute «x + 
B =e'. This results in an equation of the same order but with constant coefficients, 
which can be solved by the methods of section 15.1. If the ODE is of the Euler 
form (15.38) with a non-zero RHS then substitute x = e'; this again leads to an 
equation of the same order but with constant coefficients. If, however, f(x) = 0 in 
the Euler equation (15.38) then the equation may also be solved by substituting 
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y =x". This leads to an algebraic equation whose solution gives the allowed values 
of 4; the general solution is then the linear superposition of these functions. 
15.2.2 Exact equations 


Sometimes an ODE may be merely the derivative of another ODE of one order 
lower. If this is the case then the ODE is called exact. The nth-order linear ODE 


a” d : 
GIS + + aS + aolxy = F009, (15.41) 
dx" dx 
is exact if the LHS can be written as a simple derivative, ie. if 
d" 1 a 
AJ +o Hagley = |b eR $e + boxy] (15.42) 
dx" dx dx"—1 
It may be shown that, for (15.42) to hold, we require 
dag (x) — a(x) + a(x) — ++» + (—1)"al(xe) = 0, (15.43) 


where the prime again denotes differentiation with respect to x. If (15.43) is 
satisfied then straightforward integration leads to a new equation of one order 
lower. If this simpler equation can be solved then a solution to the original 
equation is obtained. Of course, if the above process leads to an equation that is 
itself exact then the analysis can be repeated to reduce the order still further. 


> Solve ? 4 
DCE Ms 

(l-x m2 3x7 p= il. (15.44) 
Comparing with (15.41), we have ay = 1 — x*, a, = —3x and ay = —1. It is easily shown 
that ao — a, + ay = 0, so (15.44) is exact and can therefore be written in the form 

d dy 

& | bi + bo(x)y| = 1. (15.45) 

dx dx 
Expanding the LHS of (15.45) we find 

d (, dy Py dy |), 


Comparing (15.44) and (15.46) we find 
by = 1—~x’, bi + bo = —3x, bp =—1. 


These relations integrate consistently to give b} = 1 — x? and by = —x, so (15.44) can be 
written as d di 
2 ) 
ae la x Vx »y| 1. (15.47) 


Integrating (15.47) gives us directly the first-order linear ODE 
dy ( x ) —  xte 
dx 1— x? 1— x?’ 


which can be solved by the method of subsection 14.2.4 and has the solution 


Pee | 
cysin  X+ C2 
ja ee 


V1— x? 
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It is worth noting that, even if a higher-order ODE is not exact in its given form, 
it may sometimes be made exact by multiplying through by some suitable function, 
an integrating factor, cf. subsection 14.2.3. Unfortunately, no straightforward 
method for finding an integrating factor exists and one often has to rely on 
inspection or experience. 


It is easily shown that (15.48) is not exact, but we also see immediately that by multiplying 
it through by 1/x we recover (15.44), which is exact and is solved above. < 


Another important point is that an ODE need not be linear to be exact, 
although no simple rule such as (15.43) exists if it is not linear. Nevertheless, it is 
often worth exploring the possibility that a non-linear equation is exact, since it 
could then be reduced in order by one and may lead to a soluble equation. This 
is discussed further in subsection 15.3.3. 


Solution method. For a linear ODE of the form (15.41) check whether it is exact 
using equation (15.43). If it is not then attempt to find an integrating factor which 
when multiplying the equation makes it exact. Once the equation is exact write the 
LHS as a derivative as in (15.42) and, by expanding this derivative and comparing 
with the LHS of the ODE, determine the functions by(x) in (15.42). Integrate the 
resulting equation to yield another ODE, of one order lower. This may be solved or 
simplified further if the new ODE is itself exact or can be made so. 


15.2.3 Partially known complementary function 


Suppose we wish to solve the nth-order linear ODE 


4" 
ay 


Onl) 


bail + aglxdy =f (15.49) 


and we happen to know that u(x) is a solution of (15.49) when the RHS is 
set to zero, ie. u(x) is one part of the complementary function. By making the 
substitution y(x) = u(x)v(x), we can transform (15.49) into an equation of order 
n—1 in dv/dx. This simpler equation may prove soluble. 

In particular, if the original equation is of second order then we obtain 
a first-order equation in dv/dx, which may be soluble using the methods of 
section 14.2. In this way both the remaining term in the complementary function 
and the particular integral are found. This method therefore provides a useful 
way of calculating particular integrals for second-order equations with variable 
(or constant) coefficients. 
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ay 


2 + y = cosec x. 


We see that the RHS does not fall into any of the categories listed in subsection 15.1.2, 
and so we are at an initial loss as to how to find the particular integral. However, the 
complementary function of (15.50) is 


Ve(X) = cy SIN X + C2 COS x, 
and so let us choose the solution u(x) = cosx (we could equally well choose sin x) and 
make the substitution y(x) = v(x)u(x) = v(x) cos x into (15.50). This gives 
2 


d d 
cos x5 — 2sinx = = Cosec x, (15.51) 


which is a first-order linear ODE in duv/dx and may be solved by multiplying through by 
a suitable integrating factor, as discussed in subsection 14.2.4. Writing (15.51) as 


av dv cosecx 
dx? dx cosx ” 


(15.52) 


we see that the required integrating factor is given by 
exp {72 i tanx ax} = exp [2 In(cos x)] = cos’ x. 


Multiplying both sides of (15.52) by the integrating factor cos? x we obtain 


d > dvu\ | i 
Tx (88 *Gq ) = cotx, 


which integrates to give 


dv 
cos? xe = In(sinx) + ¢. 
dx 


After rearranging and integrating again, this becomes 
v= [se x In(sin x) dx + c, fi sec? x dx 
= tanxIn(sinx) —x +c; tanx +c. 
Therefore the general solution to (15.50) is given by y = uv = vcosx, Le. 
y =c,sinx +c cosx + sin x In(sin x) — x cos x, 


which contains the full complementary function and the particular integral. < 


Solution method. If u(x) is a known solution of the nth-order equation (15.49) with 
f(x) = 0, then make the substitution y(x) = u(x)v(x) in (15.49). This leads to an 
equation of order n—1 in dv/dx, which might be soluble. 
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15.2.4 Variation of parameters 


The method of variation of parameters proves useful in finding particular integrals 
for linear ODEs with variable (and constant) coefficients. However, it requires 
knowledge of the entire complementary function, not just of one part of it as in 
the previous subsection. 
Suppose we wish to find a particular integral of the equation 
in 
ay(x)o oe + a(x) = + ag(x)y = F(X), (15.53) 


dx" 


and the complementary function y,(x) (the general solution of (15.53) with 
f(x) =0) is 
V(X) = c1yi(X) + €2Y2(x) Stee se CnYn(X), 


where the functions y,,(x) are known. We now assume that a particular integral of 
(15.53) can be expressed in a form similar to that of the complementary function, 
but with the constants c,, replaced by functions of x, Le. we assume a particular 
integral of the form 


p(X) = ki(x)yi(x) + ko(x)y2(x) + +++ + kn() Yn). (15.54) 


This will no longer satisfy the complementary equation (i.e. (15.53) with the RHS 
set to zero) but might, with suitable choices of the functions k;(x), be made equal 
to f(x), thus producing not a complementary function but a particular integral. 

Since we have n arbitrary functions k;(x), k2(x),...,kn(x), but only one restric- 
tion on them (namely the ODE), we may impose a further n — 1 constraints. We 
can choose these constraints to be as convenient as possible, and the simplest 
choice is given by 


Ki (x)yi(x) + k(x)y2(x) + + + K(x) nlx) = 0 
K(x) yx) + (x)yg(x) + + K(x) y, (0) = 0 


(15.55) 
4 (xx) yx) + kA (e) yx) + + A(x) ?(x) = 0 


“(xy D(x) ae ki (x)y"-) (x) hs gan ae Ki (xy D(x) = f(x) 


: ay(x)’ 


where the primes denote differentiation with respect to x. The last of these 
equations is not a freely chosen constraint; given the previous n — 1 constraints 
and the original ODE, it must be satisfied. 

This choice of constraints is easily justified (although the algebra is quite 
messy). Differentiating (15.54) with respect to x, we obtain 


where, for the moment, we drop the explicit x-dependence of these functions. Since 
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we are free to choose our constraints as we wish, let us define the expression in 
parentheses to be zero, giving the first equation in (15.55). Differentiating again 
we find 

Vp = kayt thay + +++ t+ kayn + ey + hays + + kv]. 


Once more we can choose the expression in brackets to be zero, giving the second 
equation in (15.55). We can repeat this procedure, choosing the corresponding 
expression in each case to be zero. This yields the first n — 1 equations in (15.55). 
The mth derivative of yp for m <n is then given by 


ee ky” + koy$ o pane kay”. 
Differentiating y, once more we find that its nth derivative is given by 
Vp? = kayy? + kay toes + kyl? + Ley + kay? $e + Ry? J. 


Substituting the expressions for ys”, m = 0 to n, into the original ODE (15.53), 
we obtain 


n 


Yo aml ky” + kayy” +e +key? Lan ey? +R +e KVP] = F(3), 
m=0 
i.e. 

n n 

Yn So kay + nl ks’ yy? + heal yO? +++ + ey’ yD] = F(X). 

m=0 j=l 


Rearranging the order of summation on the LHS, we find 


1 (n—1) 


n 
So kilanyy? +--+ + ary) toys] taal kyl? + ke’ yl? + kyl) ] = f (2. 
j=l (15.56) 


But since the functions y; are solutions of the complementary equation of (15.53) 
we have (for all j) 
any"? + iets tary; + aoyj = 0. 
Therefore (15.56) becomes 
ant yy) + ha'yg) +o + Kye? ] = fo), 


which is the final equation given in (15.55). 

Considering (15.55) to be a set of simultaneous equations in the set of unknowns 
kj (x), k5,...,k),(x), we see that the determinant of the coefficients of these functions 
is equal to the Wronskian W(y1, yo,..-,¥n), Which is non-zero since the solutions 
Ym(x) are linearly independent; see equation (15.6). Therefore (15.55) can be solved 
for the functions k/,(x), which in turn can be integrated, setting all constants of 
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integration equal to zero, to give k,,(x). The general solution to (15.53) is then 
given by 
n 
V(e) = yelx) + ype) = So Lem + km] Yl). 
m=1 

Note that if the constants of integration are included in the k,,(x) then, as well 
as finding the particular integral, we redefine the arbitrary constants c,, in the 
complementary function. 


> Use the variation-of-parameters method to solve 


a 
= + y = cosec x, (15.57) 


subject to the boundary conditions y(0) = y(x/2) = 0. 


The complementary function of (15.57) is again 
Ve(X) = cy SIN X + C2 COS x. 


We therefore assume a particular integral of the form 


Yp(x) = k(x) sin x + k2(x) cos x, 
and impose the additional constraints of (15.55), ie. 
ki (x) sin x + k}(x) cos x = 0, 
ki (x) cos x — k5(x) sin x = cosec x. 
Solving these equations for k{(x) and k}(x) gives 
k(x) = cos x cosec x = cot x, 
k(x) = — sin x cosecx = —1. 
Hence, ignoring the constants of integration, k,(x) and k2(x) are given by 
k,(x) = In(sin x), 
k(x) = —x. 
The general solution to the ODE (15.57) is therefore 
y(x) = [c1 + In(sin x)] sin x + (c2 — x) cos x, 


which is identical to the solution found in subsection 15.2.3. Applying the boundary 
conditions y(0) = y(z/2) = 0 we find c; = c. = 0 and so 


y(x) = In(sin x) sin x — xcosx. <4 


Solution method. If the complementary function of (15.53) is known then assume 
a particular integral of the same form but with the constants replaced by functions 
of x. Impose the constraints in (15.55) and solve the resulting system of equations 


for the unknowns k}(x),k4,...,k}(x). Integrate these functions, setting constants of 
integration equal to zero, to obtain k,(x),k2(x),...,k,(x) and hence the particular 
integral. 
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15.2.5 Green’s functions 


The Green’s function method of solving linear ODEs bears a striking resemblance 
to the method of variation of parameters discussed in the previous subsection; 
it too requires knowledge of the entire complementary function in order to find 
the particular integral and therefore the general solution. The Green’s function 
approach differs, however, since once the Green’s function for a particular LHS 
of (15.1) and particular boundary conditions has been found, then the solution 
for any RHS (ie. any f(x)) can be written down immediately, albeit in the form 
of an integral. 

Although the Green’s function method can be approached by considering the 
superposition of eigenfunctions of the equation (see chapter 17) and is also 
applicable to the solution of partial differential equations (see chapter 21), this 
section adopts a more utilitarian approach based on the properties of the Dirac 
delta function (see subsection 13.1.3) and deals only with the use of Green’s 
functions in solving ODEs. 

Let us again consider the equation 


n 


iE ees ene Aap SGC), (15.58) 


dx" dx 


but for the sake of brevity we now denote the LHS by Ly(x), Le. as a linear 
differential operator acting on y(x). Thus (15.58) now reads 


Ly(x) = f(x). (15.59) 


Let us suppose that a function G(x, z) (the Green’s function) exists such that the 
general solution to (15.59), which obeys some set of imposed boundary conditions 
in the range a < x < J, is given by 


b 
vo) = f G(x, z) f(z) dz, (15.60) 


where z is the integration variable. If we apply the linear differential operator £ 
to both sides of (15.60) and use (15.59) then we obtain 


b 
Lyx) = f [LG(x,z)] f(z) dz = f(x). (15.61) 


Comparison of (15.61) with a standard property of the Dirac delta function (see 
subsection 13.1.3), namely 


b 
fics) =f 8(e—2)f16) de, 


for a< x <b, shows that for (15.61) to hold for any arbitrary function f(x), we 
require (for a < x < b) that 


LG(x,z) = 5(x —z), (15.62) 
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ie. the Green’s function G(x,z) must satisfy the original ODE with the RHS set 
equal to a delta function. G(x,z) may be thought of physically as the response of 
a system to a unit impulse at x = z. 

In addition to (15.62), we must impose two further sets of restrictions on 
G(x, z). The first is the requirement that the general solution y(x) in (15.60) obeys 
the boundary conditions. For homogeneous boundary conditions, in which y(x) 
and/or its derivatives are required to be zero at specified points, this is most 
simply arranged by demanding that G(x, z) itself obeys the boundary conditions 
when it is considered as a function of x alone; if, for example, we require 
y(a) = y(b) = 0 then we should also demand G(a,z) = G(b,z) = 0. Problems 
having inhomogeneous boundary conditions are discussed at the end of this 
subsection. 

The second set of restrictions concerns the continuity or discontinuity of G(x, z) 
and its derivatives at x = z and can be found by integrating (15.62) with respect 
to x over the small interval [z — e, z +e] and taking the limit as e — 0. We then 
obtain 


. jock 2) = Z+E . 
Hig an a= tim [ 6(x —z)dx =1. (15.63) 


Since d"G/dx" exists at x = z but with value infinity, the (n—1)th-order derivative 
must have a finite discontinuity there, whereas all the lower-order derivatives, 
d"G/dx" for m < n—1, must be continuous at this point. Therefore the terms 
containing these derivatives cannot contribute to the value of the integral on 
the LHS of (15.63). Noting that, apart from an arbitrary additive constant, 
[(d"G/dx") dx = d"'G/dx"", and integrating the terms on the LHS of (15.63) 
by parts we find 

Z+E d™G(x, z) 


lim An(X) 


60 J, dx™ 


dx =0 (15.64) 


for m = 0 to n— 1. Thus, since only the term containing d"G/dx" contributes to 
the integral in (15.63), we conclude, after performing an integration by parts, that 


n—1 z+e 
od =e (15.65) 


dx"! 


lim ato) a 
Thus we have the further n constraints that G(x, z) and its derivatives up to order 
n—2 are continuous at x = z but that d’-'G/dx"—! has a discontinuity of 1/a,(z) 
atx =z. 

Thus the properties of the Green’s function G(x, z) for an nth-order linear ODE 
may be summarised by the following. 


(i) G(x,z) obeys the original ODE but with f(x) on the RHS set equal to a 
delta function 6(x — z). 
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(ii) When considered as a function of x alone G(x,z) obeys the specified 
(homogeneous) boundary conditions on y(x). 

(iii) The derivatives of G(x, z) with respect to x up to order n—2 are continuous 
at x = z, but the (n — 1)th-order derivative has a discontinuity of 1/a,(z) 
at this point. 


> Use Green’s functions to solve 


a 
me + y =cosec x, (15.66) 


subject to the boundary conditions y(0) = y(x/2) = 0. 


From (15.62) we see that the Green’s function G(x,z) must satisfy 


@G(x,z) 
dx? 


Now it is clear that for x # z the RHS of (15.67) is zero, and we are left with the 
task of finding the general solution to the homogeneous equation, i.e. the complementary 
function. The complementary function of (15.67) consists of a linear superposition of sin x 
and cosx and must consist of different superpositions on either side of x = z, since its 
(n — 1)th derivative (i.e. the first derivative in this case) is required to have a discontinuity 
there. Therefore we assume the form of the Green’s function to be 


+ G(x, z) = 0(x — z). (15.67) 


G(x,2) A(z)sinx + B(z)cosx for x <z, 

ape C(z)sinx + D(z)cosx for x >z. 

Note that we have performed a similar (but not identical) operation to that used in the 
variation-of-parameters method, i.e. we have replaced the constants in the complementary 
function with functions (this time of z). 

We must now impose the relevant restrictions on G(x,z) in order to determine the 
functions A(z),...,D(z). The first of these is that G(x, z) should itself obey the homogeneous 
boundary conditions G(0,z) = G(z/2,z) = 0. This leads to the conclusion that B(z) = 
C(z) = 0, so we now have 

A(z)sinx for x <z, 
G(x, 2) = 


D(z)cosx for x >z. 


The second restriction is the continuity conditions given in equations (15.64), (15.65), 
namely that, for this second-order equation, G(x,z) is continuous at x = z and dG/dx has 
a discontinuity of 1/a,(z) = 1 at this point. Applying these two constraints we have 


D(z)cosz — A(z)sinz =0 
—D(z)sinz — A(z)cosz = 1. 


Solving these equations for A(z) and D(z), we find 
A(z) = —cosz, D(z) =—sinz. 


Thus we have 
—coszsinx forx <z, 
—sinzcosx forx>z. 


G(x,z) = { 
Therefore, from (15.60), the general solution to (15.66) that obeys the boundary conditions 
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y(0) = y(x/2) = 0 is given by 


n/2 
y(x) = | G(x, z) cosec z dz 
0 


x n/2 
= —cosx [ sin z cosec z dz — sinx | cos z cosec z dz 
0 x 


=—xcosx + sin x In(sin x), 


which agrees with the result obtained in the previous subsections. < 


As mentioned earlier, once a Green’s function has been obtained for a given 
LHS and boundary conditions, it can be used to find a general solution for any 
RHS; thus, the solution of d?y/dx? + y = f(x), with y(0) = y(a/2) = 0, is given 
immediately by 


n/2 
vos) = [ Gtx2)fte)dz 
x n/2 
= —eosx | sinz f(z)dz — sin | cosz f(z) dz. (15.68) 


As an example, the reader may wish to verify that if f(x) = sin 2x then (15.68) 
gives y(x) = (—sin2x)/3, a solution easily verified by direct substitution. In 
general, analytic integration of (15.68) for arbitrary f(x) will prove intractable; 
then the integrals must be evaluated numerically. 

Another important point is that although the Green’s function method above 
has provided a general solution, it is also useful for finding a particular integral 
if the complementary function is known. This is easily seen since in (15.68) the 
constant integration limits 0 and 2/2 lead merely to constant values by which 
the factors sinx and cosx are multiplied; thus the complementary function is 
reconstructed. The rest of the general solution, i.e. the particular integral, comes 
from the variable integration limit x. Therefore by changing ce to — {*, and so 
dropping the constant integration limits, we can find just the particular integral. 
For example, a particular integral of d?y/dx? + y = f(x) that satisfies the above 
boundary conditions is given by 


x 


Yp(x) = — cosx | sinz f(z)dz+ sinx | cosz f(z) dz. 


A very important point to realise about the Green’s function method is that a 
particular G(x,z) applies to a given LHS of an ODE and the imposed boundary 
conditions, i.e. the same equation with different boundary conditions will have a 
different Green’s function. To illustrate this point, let us consider again the ODE 
solved in (15.68), but with different boundary conditions. 
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> Use Green’s functions to solve 


@y 


pa + Y = FO) 


subject to the one-point boundary conditions y(0) = y'(0) = 0. 


We again require (15.67) to hold and so again we assume a Green’s function of the form 


A(z)sinx + B(z)cosx for x < z, 


G(x,z) = { 


C(z)sinx + D(z)cosx for x >z. 


However, we now require G(x,z) to obey the boundary conditions G(0,z) = G’(0,z) = 0, 
which imply A(z) = B(z) = 0. Therefore we have 
0 for x < z, 


G(x, Zz) = { 


C(z)sinx + D(z)cosx for x >z. 
Applying the continuity conditions on G(x,z) as before now gives 
C(z)sinz + D(z)cosz = 0, 
C(z)cosz — D(z) sinz = 1, 
which are solved to give 
C(z) = cosz, D(z) =—sinz. 
So finally the Green’s function is given by 


for x < z, 


0 
G(x,z) = { 


sin(x—z) forx>z, 


and the general solution to (15.69) that obeys the boundary conditions y(0) = y’(0) = 0 is 
vox) =f Gls,2ifle)ae 
0 
-| sin(x — z)f(z)dz. < 
0 


Finally, we consider how to deal with inhomogeneous boundary conditions 
such as y(a) = 4, y(b) = B or y(0) = y'(0) = y, where a, B, y are non-zero. The 
simplest method of solution in this case is to make a change of variable such that 
the boundary conditions in the new variable, u say, are homogeneous, 1.e. u(a) = 
u(b) = 0 or u(0) = u'(0) = 0 etc. For nth-order equations we generally require 
n boundary conditions to fix the solution, but these n boundary conditions can 
be of various types: we could have the n-point boundary conditions y(Xm) = Vm 
for m = 1 to n, or the one-point boundary conditions y(xo) = y'(xo) =--- = 
y"-))(xo) = yo, or something in between. In all cases a suitable change of variable 
is 


u=y—h(x), 


where h(x) is an (n — 1)th-order polynomial that obeys the boundary conditions. 
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For example, if we consider the second-order case with boundary conditions 
y(a) = a, y(b) = B then a suitable change of variable is 


u=y—(mx +c), 


where y = mx +c is the straight line through the points (a, «) and (b, 6), for which 
m = (a — B)/(a—b) and c = (Ba — xb)/(a — b). Alternatively, if the boundary 
conditions for our second-order equation are y(0) = y’(0) = y then we would 
make the same change of variable, but this time y = mx +c would be the straight 
line through (0,y) with slope y, ie.m=c=y. 


Solution method. Require that the Green’s function G(x, z) obeys the original ODE, 
but with the RHS set to a delta function 6(x — z). This is equivalent to assuming 
that G(x,z) is given by the complementary function of the original ODE, with the 
constants replaced by functions of z; these functions are different for x < z and x > 
z. Now require also that G(x,z) obeys the given homogeneous boundary conditions 
and impose the continuity conditions given in (15.64) and (15.65). The general 
solution to the original ODE is then given by (15.60). For inhomogeneous boundary 
conditions, make the change of dependent variable u = y — h(x), where h(x) is a 
polynomial obeying the given boundary conditions. 


15.2.6 Canonical form for second-order equations 
In this section we specialise from nth-order linear ODEs with variable coefficients 
to those of order 2. In particular we consider the equation 
@y 
+ ary = + ao(x)y = f(x), (15.70) 


which has been rearranged so that the coefficient of d*y/dx? is unity. By making 
the substitution y(x) = u(x)v(x) we obtain 


Fa A y a 
"+ (= +a1) fe. (“ Bee = eer) pal, (15.71) 
Uu 


u u 


where the prime denotes differentiation with respect to x. Since (15.71) would be 
much simplified if there were no term in v’, let us choose u(x) such that the first 
factor in parentheses on the LHS of (15.71) is zero, i.e. 


ae u(x) = exp {— 4 faved r} (15.72) 


We then obtain an equation of the form 
2y 
Te + g(x)v = h(x), (15.73) 
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where 
a(x) = do(x) — Fla? — 544 (x) 
h(x) = f(x) exp {+f auevas} ; 


Since (15.73) is of a simpler form than the original equation, (15.70), it may 
prove easier to solve. 


> Solve 


+(x? —1)y =0. (15.74) 


Dividing (15.74) through by 4x7, we see that it is of the form (15.70) with a;(x) = 1/x, 
do(x) = (x? — 1)/4x? and f(x) = 0. Therefore, making the substitution 


y= ou =vexp [u«)-% 
le ah eed a 


we obtain 
fv ov 
—+-=0. 15.75 
dx? a 4 ( ) 
Equation (15.75) is easily solved to give 
v=c,sin $x +c) cos 5x, 
so the solution of (15.74) is 


v c; sin sx te cos 5x 
y= SS = 


As an alternative to choosing u(x) such that the coefficient of v’ in (15.71) is 
zero, we could choose a different u(x) such that the coefficient of v vanishes. For 
this to be the case, we see from (15.71) that we would require 


u” + au’ + agu = 0, 


so u(x) would have to be a solution of the original ODE with the RHS set to 
zero, 1.e. part of the complementary function. If such a solution were known then 
the substitution y = uv would yield an equation with no term in v, which could 
be solved by two straightforward integrations. This is a special (second-order) 
case of the method discussed in subsection 15.2.3. 


Solution method. Write the equation in the form (15.70), then substitute y = uv, 
where u(x) is given by (15.72). This leads to an equation of the form (15.73), in 
which there is no term in dv/dx and which may be easier to solve. Alternatively, 
if part of the complementary function is known then follow the method of subsec- 
tion 15.2.3. 
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15.3 General ordinary differential equations 


In this section, we discuss miscellaneous methods for simplifying general ODEs. 
These methods are applicable to both linear and non-linear equations and in 
some cases may lead to a solution. More often than not, however, finding a 
closed-form solution to a general non-linear ODE proves impossible. 


15.3.1 Dependent variable absent 


If an ODE does not contain the dependent variable y explicitly, but only its 
derivatives, then the change of variable p = dy/dx leads to an equation of one 
order lower. 


> Solve 


Py ,,dy 
EN P= ok 15.76 
dx? a dx % ( ) 


This is transformed by the substitution p = dy/dx to the first-order equation 


uns 2p = 4x. (15.77) 
dx 


The solution to (15.77) is then found by the method of subsection 14.2.4 and reads 


dy 


= Fy ae + 2x-1, 


where a is a constant. Thus by direct integration the solution to the original equation, 
(15.76), is 


y(x) = ce +x° —x+co. < 


An extension to the above method is appropriate if an ODE contains only 
derivatives of y that are of order m and greater. Then the substitution p = d"y/dx™ 
reduces the order of the ODE by m. 


Solution method. If the ODE contains only derivatives of y that are of order m and 
greater then the substitution p = d"y/dx"™ reduces the order of the equation by m. 


15.3.2 Independent variable absent 


If an ODE does not contain the independent variable x explicitly, except in d/dx, 
d? /dx? etc., then as in the previous subsection we make the substitution p = dy/dx 
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but also write 


ay _ dp _dydp __dp 
dx? dx dxdy dy 
B d ( dp\ dy d (dp 5@p dp\? 

a2 = = [p= |) = = pe) a pte — 15.78 
a dx (o£) dx dy (of) Rr +0(2) be ee 
and so on for higher-order derivatives. This leads to an equation of one order 
lower. 


pos 


> Solve 


d 2 
1 ayes () =a (15.79) 


Making the substitutions dy/dx = p and d?y/dx? = p(dp/dy) we obtain the first-order 
ODE 


d 
1+ pq +P =0, 


which is separable and may be solved as in subsection 14.2.1 to obtain 


(1+p*)y’ = a1. 
Using p = dy/dx we therefore have 
dy cj-y? 
i ga 1 
PS ae ae 


which may be integrated to give the general solution of (15.79); after squaring this reads 


(xtoyr+y =c. < 


Solution method. If the ODE does not contain x explicitly then substitute p = 
dy/dx, along with the relations for higher derivatives given in (15.78), to obtain an 
equation of one order lower, which may prove easier to solve. 


15.3.3 Non-linear exact equations 


As discussed in subsection 15.2.2, an exact ODE is one that can be obtained by 
straightforward differentiation of an equation of one order lower. Moreover, the 
notion of exact equations is useful for both linear and non-linear equations, since 
an exact equation can be immediately integrated. It is possible, of course, that 
the resulting equation may itself be exact, so that the process can be repeated. 
In the non-linear case, however, there is no simple relation (such as (15.43) for 
the linear case) by which an equation can be shown to be exact. Nevertheless, a 
general procedure does exist and is illustrated in the following example. 
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Directing our attention to the term on the LHS of (15.80) that contains the highest-order 
derivative, ic. 2y d3y/dx3, we see that it can be obtained by differentiating 2y d*y/dx* since 


d @y dy dy d’y 
x (3) = ry 422 oe. (15.81) 


Rewriting the LHS of (15.80) using (15.81), we are left with 4(dy/dx)(d*y/dy’), which may 
itself be written as a derivative, ie. 


dy@y  d dy\? 
4 ar 2 ae : (15.82) 
Since, therefore, we can write the LHS of (15.80) as a sum of simple derivatives of other 


functions, (15.80) is exact. Integrating (15.80) with respect to x, and using (15.81) and 
(15.82), now gives 


@y dy\? x? 
5h +2(2) [sax 5 te: (15.83) 


Now we can repeat the process to find whether (15.83) is itself exact. Considering the term 
on the LHS of (15.83) that contains the highest-order derivative, ie. 2y d*y/dx*, we note 
that we obtain this by differentiating 2y dy/dx, as follows: 


The above expression already contains all the terms on the LHS of (15.83), so we can 
integrate (15.83) to give 
y x3 
ae sre +ox+c. 


Integrating once more we obtain the solution 
ot cat 
y = 74 5) +Ox+c3. <4 
It is worth noting that both linear equations (as discussed in subsection 15.2.2) 
and non-linear equations may sometimes be made exact by multiplying through 
by an appropriate integrating factor. Although no general method exists for 
finding such a factor, one may sometimes be found by inspection or inspired 


guesswork. 


Solution method. Rearrange the equation so that all the terms containing y or its 
derivatives are on the LHS, then check to see whether the equation is exact by 
attempting to write the LHS as a simple derivative. If this is possible then the 
equation is exact and may be integrated directly to give an equation of one order 
lower. If the new equation is itself exact the process can be repeated. 
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15.3.4 Isobaric or homogeneous equations 


It is straightforward to generalise the discussion of first-order isobaric equations 
given in subsection 14.2.6 to equations of general order n. An nth-order isobaric 
equation is one in which every term can be made dimensionally consistent upon 
giving y and dy each a weight m, and x and dx each a weight 1. Then the nth 
derivative of y with respect to x, for example, would have dimensions m in y 
and —n in x. In the special case m = 1, for which the equation is dimensionally 
consistent, the equation is called homogeneous (not to be confused with linear 
equations with a zero RHS). If an equation is isobaric or homogeneous then the 
change in dependent variable y = vx” (y = vx in the homogeneous case) followed 
by the change in independent variable x = e' leads to an equation in which the 
new independent variable t is absent except in the form d/dt. 


> Solve 
+(y? + xy) =0. (15.84) 


Assigning y and dy the weight m, and x and dx the weight 1, the weights of the five terms 
on the LHS of (15.84) are, from left to right: m+ 1, m+ 1, 2m, 2m, m+ 1. For these 
weights all to be equal we require m = 1; thus (15.84) is a homogeneous equation. Since it 
is homogeneous we now make the substitution y = vx, which, after dividing the resulting 
equation through by x3, gives 


ov dv 
xa + (1 — vo) =0. (15.85) 

Now substituting x = e' into (15.85) we obtain (after some working) 

vo dv 

— —v— = 0. 15.86 

ae ( ) 
which can be integrated directly to give 

dv 

ae 30° +¢1. (15.87) 


Equation (15.87) is separable, and integrates to give 


Rearranging and using x = e' and y = vx we finally obtain the solution to (15.84) as 
y=d,xtan (S41 Inx + did) _< 


Solution method. Assume that y and dy have weight m, and x and dx weight 1, 
and write down the combined weights of each term in the ODE. If these weights can 
be made equal by assuming a particular value for m then the equation is isobaric 
(or homogeneous if m = 1). Making the substitution y = vx" followed by x = e' 
leads to an equation in which the new independent variable t is absent except in the 
form d/dt. 
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15.3.5 Equations homogeneous in x or y alone 


It will be seen that the intermediate equation (15.85) in the example of the 
previous subsection was simplified by the substitution x = e', in that this led to 
an equation in which the new independent variable t occurred only in the form 
d/dt; see (15.86). A closer examination of (15.85) reveals that it is dimensionally 
consistent in the independent variable x taken alone; this is equivalent to giving 
the dependent variable and its differential a weight m = 0. For any equation that 
is homogeneous in x alone, the substitution x = e' will lead to an equation that 
does not contain the new independent variable t except as d/dt. Note that the 
Euler equation of subsection 15.2.1 is a special, linear example of an equation 
homogeneous in x alone. Similarly, if an equation is homogeneous in y alone, then 
substituting y = e” leads to an equation in which the new dependent variable, v, 
occurs only in the form d/dv. 


> Solve 


This equation is homogeneous in x alone, and on substituting x = e' we obtain 
BV. 2 
dt? yp 

which does not contain the new independent variable t except as d/dt. Such equations 


may often be solved by the method of subsection 15.3.2, but in this case we can integrate 
directly to obtain 


=0, 


dy 
a = V2e + 1/9). 
This equation is separable, and we find 
i —— a =t+c. 
J 2Xe1 +1/y?) 


By multiplying the numerator and denominator of the integrand on the LHS by y, we find 
the solution 


vay +1 
= [+e 
2c 
Remembering that t = Inx, we finally obtain 
Vayr+1 
oP ais =Inx+co. <4 


V2e1 


Solution method. If the weight of x taken alone is the same in every term in the 
ODE then the substitution x = e' leads to an equation in which the new independent 
variable t is absent except in the form d/dt. If the weight of y taken alone is the 
same in every term then the substitution y = e’ leads to an equation in which the 
new dependent variable v is absent except in the form d/dv. 
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15.3.6 Equations having y = Ae* as a solution 


Finally, we note that if any general (linear or non-linear) nth-order ODE is 
satisfied identically by assuming that 
dy d"y 
Pept pee mie eee 15.88 
Y ax dx" cee) 
then y = Ae* is a solution of that equation. This must be so because y = Ae* is 
a non-zero function that satisfies (15.88). 


> Find a solution of 


2) 
Cee eres «() 0. (15.89) 


Setting y = dy/dx = d?y/dx* in (15.89), we obtain 
(x? + x)y? — x’y? — xy? =0, 


which is satisfied identically. Therefore y = Ae* is a solution of (15.89); this is easily 
verified by directly substituting y = Ae* into (15.89). < 


Solution method. If the equation is satisfied identically by making the substitutions 
y =dy/dx =---=d"y/dx" then y = Ae* is a solution. 


15.4 Exercises 


15.1 A simple harmonic oscillator, of mass m and natural frequency wo, experiences 
an oscillating driving force f(t) = macos ot. Therefore, its equation of motion is 
ax a 
me + @ox = acos wt, 
where x is its position. Given that at t = 0 we have x = dx/dt = 0, find the 
function x(t). Describe the solution if m is approximately, but not exactly, equal 


to Wo. 
15.2 Find the roots of the auxiliary equation for the following. Hence solve them for 
the boundary conditions stated. 
vf df 1 
(a) re Pee +5f=0, with f(0) =1,f'(0) =0. 
2 
(b) at 42h +5f =e'cos3t, with f(0) = 0, f’(0) =0. 


15.3 The theory of bent beams shows that at any point in the beam the ‘bending 
moment’ is given by K/p, where K is a constant (that depends upon the beam 
material and cross-sectional shape) and p is the radius of curvature at that point. 
Consider a light beam of length L whose ends, x = 0 and x = L, are supported 
at the same vertical height and which has a weight W suspended from its centre. 
Verify that at any point x (0 < x < L/2 for definiteness) the net magnitude of 
the bending moment (bending moment = force x perpendicular distance) due to 
the weight and support reactions, evaluated on either side of x, is Wx/2. 
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15.4 


15.6 


15.8 


If the beam is only slightly bent, so that (dy/dx)*? < 1, where y = y(x) is the 
downward displacement of the beam at x, show that the beam profile satisfies 
the approximate equation 

Py Wx 

dx? 2K" 
By integrating this equation twice and using physically imposed conditions on 
your solution at x = 0 and x = L/2, show that the downward displacement at 
the centre of the beam is WL?/(48K). 
Solve the differential equation 


subject to the conditions f = 0 and df/dt = 2 at t=0. 

Find the equation satisfied by the positions of the turning points of f(t) and 
hence, by drawing suitable sketch graphs, determine the number of turning points 
the solution has in the range t > 0 if (a) 2 = 1/4, and (b) 2 = —1/4. 

The function f(t) satisfies the differential equation 
Pf df 
8— 
ae a dt 
For the following sets of boundary conditions determine whether it has solutions, 
and, if so, find them: 


(a) f) =0, f"(0)=0, fn 3) =0 
(b) f)=0, f(0)=-2, fn 2) = 


Determine the values of « and f for which the following four functions are 
linearly dependent: 


+12f =12e. 


yi(x) = x cosh x + sinh x, 


y3(x) = (x + a)e* 
vax) = (x + B Ne o 
You will find it convenient to work with those linear combinations of the y;(x) 


that can be written the most compactly. 
A solution of the differential equation 


y= 

yo(x) = x sinh x + cosh x, 
y= 
)= 


' when x = 1. What is its value 


takes the value 1 when x = 0 and the value e~ 
when x = 2? 


The two functions x(t) and y(t) satisfy the simultaneous equations 


dx 


a —2y=-sint, 


By. + 2x = 5cost. 

dt 
Find explicit expressions for x(t) and y(t), given that x(0) = 3 and y(0) = 2. 
Sketch the solution trajectory in the xy-plane for 0 < t < 22, showing that 
the trajectory crosses itself at (0,1/2) and passes through the points (0,—3) and 
(0,—1) in the negative x-direction. 


524 


15.4 EXERCISES 


15.9 


15.10 


15.11 


15.12 


15.13 


15.14 


15.15 


Find the general solutions of 
a d 
(a) 12 + 1l6y = 32x —8, 


x 2 
(b) d (<2) + (2acoth 2ax) (=) = 2a’, 
y dx 


where a is a constant. 
Use the method of Laplace transforms to solve 


2 
@ Hrslisp=0, fO=1, /O=-4 


2 
(b) af. 44f 


422 45f=0, fO=1L sO) =0. 
The quantities x(t), y(t) satisfy the simultaneous equations 


de | dt 


x +2nx +n?x =0, 
P+ ny +n’y = px, 
where x(0) = y(0) = (0) = 0 and x(0) = 4. Show that 
y(t) = Sat? (1 — 4nt) exp(—nt). 


Use Laplace transforms to solve, for t > 0, the differential equations 


X+2x+y=cost, 
y+2x+3y =2cost, 


which describe a coupled system that starts from rest at the equilibrium position. 
Show that the subsequent motion takes place along a straight line in the xy-plane. 
Verify that the frequency at which the system is driven is equal to one of the 
resonance frequencies of the system; explain why there is no resonant behaviour 
in the solution you have obtained. 
Two unstable isotopes A and B and a stable isotope C have the following decay 
rates per atom present: A > B, 3s-!; A> C, 1s"'; B > C, 2s". Initially a 
quantity x9 of A is present, but there are no atoms of the other two types. Using 
Laplace transforms, find the amount of C present at a later time t. 
For a lightly damped (y < mo) harmonic oscillator driven at its undamped 
resonance frequency wo, the displacement x(t) at time f satisfies the equation 
2 

= + 29S + max = F sin wot. 
Use Laplace transforms to find the displacement at a general time if the oscillator 
starts from rest at its equilibrium position. 


(a) Show that ultimately the oscillation has amplitude F/(2moy), with a phase 
lag of 2/2 relative to the driving force per unit mass F. 

(b) By differentiating the original equation, conclude that if x(t) is expanded as 
a power series in t for small t, then the first non-vanishing term is Foot? /6. 
Confirm this conclusion by expanding your explicit solution. 


The ‘golden mean’, which is said to describe the most aesthetically pleasing 
proportions for the sides of a rectangle (e.g. the ideal picture frame), is given 
by the limiting value of the ratio of successive terms of the Fibonacci series uy, 
which is generated by 


Un42 = Und + Un, 


with uo = 0 and u, = 1. Find an expression for the general term of the series and 
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15.16 


15.17 


15.18 


15.19 


15.20 


15.21 


15.22 


verify that the golden mean is equal to the larger root of the recurrence relation’s 
characteristic equation. 
In a particular scheme for numerically modelling one-dimensional fluid flow, the 
successive values, u,, of the solution are connected for n > 1 by the difference 
equation 
CUn +1 a Un—1) = d(Uns1 — 2Un + Un—1)s 
where c and d are positive constants. The boundary conditions are uy = 0 and 
um = 1. Find the solution to the equation, and show that successive values of u, 
will have alternating signs if c > d. 
The first few terms of a series u,, starting with uo, are 1,2,2,1,6,—3. The series 
is generated by a recurrence relation of the form 
Un = Puy—2 + Qtn—a, 
where P and Q are constants. Find an expression for the general term of the 
series and show that, in fact, the series consists of two interleaved series given by 
Ym = z + 34”, 

Wam+1 = i x 34”, 
for m=0,1,2,.... 
Find an explicit expression for the u, satisfying 

Unzt + Sun + 6U,1 = 2", 


given that uy = u,; = 1. Deduce that 2” — 26(—3)" is divisible by 5 for all 
non-negative integers n. 
Find the general expression for the u, satisfying 


Uni = 2Uy—2 Un 


with uo =u; = 0 and uz = 1, and show that they can be written in the form 


s 1, “on? a 3nn é 
n 5 J5 4 + 
where tan d = 2. 


Consider the seventh-order recurrence relation 


Unt7 — Unto — Uns + Und — Ung3 + Unga + Une1 — Un = 0. 


Find the most general form of its solution, and show that: 


(a) if only the four initial values up = 0, uw; = 2, uw = 6 and uz = 12, are specified, 
then the relation has one solution that cycles repeatedly through this set of 
four numbers; 

(b) but if, in addition, it is required that us = 20, us = 30 and us = 42 then the 
solution is unique, with u, = n(n + 1). 


Find the general solution of 


given that y(1) = 1 and y(e) = 2e. 
Find the general solution of 


ay d 
(x + ys 43(x+ n> ty=x, 
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15.23 


15.24 


15.25 


15.26 


15.27 


15.28 


Prove that the general solution of 


is given by 


1 2 1 
09 Gag F(a) to]: 


Use the method of variation of parameters to find the general solutions of 
ay dy dy ; 
2 = 2xe*. 
dx? dx? dx es wn 
Use the intermediate result of exercise 15.24(a) to find the Green’s function that 
satisfies 
#G(x,€) 
dx? 


(a) 


y=x", (b) 


~G(x,é)=d(x—€) with ~=—- G(0,) = G1, ) = 0. 


Consider the equation 


dy 


ay dy 
dx 


Fx, y) = x(x + 1) +(2—x’) (2+x)y =0. 


(a) Given that y;(x) = 1/x is one of its solutions, find a second linearly inde- 
pendent one, 
(i) by setting y2(x) = y1(x)u(x), and 
(ii) by noting the sum of the coefficients in the equation. 
(b) Hence, using the variation of parameters method, find the general solution 
of 


F(x,y) =(x +1). 

Show generally that if y;(x) and y(x) are linearly independent solutions of 
dy 
dx 
with y,(0) = 0 and y2(1) = 0, then the Green’s function G(x, €) for the interval 
0< x,é < 1 and with G(0,é) = G(1,€) = 0 can be written in the form 

F vilx)yxS)/W(S) O<x <6, 

G(x, €) = ‘ ‘ 
yrx)y(/W(S) o<x<l, 

where W(x) = W[y1(x), y2(x)] is the Wronskian of y;(x) and y2(x). 


Use the result of the previous exercise to find the Green’s function G(x, €) that 
satisfies 


SF + (x) + a(x)y =0, 


in the interval 0 < x,é < 1, with G(0,é) = G(1,é) = 0. Hence obtain integral 
expressions for the solution of 


0 0<x<Xo, 


fh de ae So 
a Paced " x Se eT, 
distinguishing between the cases (a) x < xo, and (b) x > Xo. 
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15.29 


15.30 


15.31 


15.32 


15.33 


15.34 


15.35 


The equation of motion for a driven damped harmonic oscillator can be written 
X 4+ 2k + (14+ 4x7)x = f(t), 


with x # 0. If it starts from rest with x(0) = 0 and x(0) = 0, find the corresponding 
Green’s function G(t,t) and verify that it can be written as a function of t—t 
only. a Hs explicit solution when the driving force is the unit step function, 
ie. f(t) = ). Confirm your solution by taking the Laplace transforms of both 
it and the Ran equation. 

Show that the Green’s function for the equation 


subject to the boundary conditions y(0) = y(z) = 0, is given by 


{ —2 cos 
G(x, Zz) = 


3x sin $z 0<z<x, 
: 1 
—2 sin 5 


X COS 5Z 


1 
3 
; x<SZ2<70. 


Find the Green’s function x = G(t, to) that solves 


— —=d(t-t 
dt? 4 dt po) 
under the initial conditions x = dx/dt = 0 at t = 0. Hence solve 
ax dx 
+a = f(0, 


where f(t) = 0 for t <0. 
Evaluate your answer explicitly for f(t) = Ae (t > 0). 
Consider the equation 
a 
dx 


where f(y) can be any function. 


+ f(y) =0, 


(a) By multiplying through by dy/dx, obtain the general solution relating x and 
y. 
(b) A mass m, initially at rest at the point x = 0, is accelerated by a force 


f(x) = A(x — x) f +2in (1 = *) 
a) 


Its equation of motion is md?x/dt? = f(x). Find x as a function of time, and 
show that ultimately the particle has travelled a distance xo. 
Solve 
dy dy\ dy dy : 
2y$2 42(y 4a) 2 42(2) sin x. 


Find the general solution of the equation 


ay ay 
x5 +255 =A x 


Express the equation 
#y 
be 

in canonical form and hence ne its general solution. 


+ ae + (4x? + 6)y =e sin2x 
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15.36 


15.37 


15.27 


15.29 


Find the form of the solutions of the equation 
dy @y 2,,\ 2 2 
DED tof EY ef AY Ny 
dx dx3 dx? dx 
that have y(0) = 


[You will seas the result f cosech udu = —In(cosech z + cothz).] 
Consider the equation 


” n+3—2 —1 7 —2 : — n 
BE ee Gey) ae 


in which p 4 2 and n> —1 but n #1. For the boundary conditions y(1) = 0 and 
y'(1) = A, show that the solution is y(x) = v(x)x-?/"—), where v(x) is given by 


) dz 
i 17) Inx. 
0 [42 4+22"41/(n+ 1)] 


15.5 Hints and answers 


The function is a(wg — 7)~!(cos wt — cos wot); for moderate t, x(t) is a sine wave 
of linearly increasing amplitude i sin Mot)/(2@o); for large t it shows beats of 
maximum emplitnde 2(@ — @’)}” 

Ignore the term y? , compared with 1, in the expression for p. y = 0 at x = 0. 
From symmetry, dy/dx = 0 at x = L/2. 

General solution f(t) = Ae~* + Be! — 3e~*. (a) No solution, inconsistent 
boundary conditions; (b) f(t) = 2e~% + e~*' — 3e-*., 

The auxiliary equation has repeated roots and the RHS is contained in the 
complementary function. The solution is y(x) = (A+Bx)e*+2x7e™. y(2) = Se. 
(a) The auxiliary equation has roots 2, 2, —4; (A+Bx) exp 2x+C exp(—4x)+2x+1; 
(b) multiply through by sinh 2ax and note that 

f cosech 2ax dx = (2a)! In(| tanh ax|); y = B(sinh 2ax)'/?(| tanh ax|)4. 

Use Laplace transforms; write s(s-+n)~* as (s +n) —n(s + n)~*. 

L [C(t)] = xo(s + 8)/[s(s + 2)(s + 4)], yielding 

C(t) = xof1 + + exp(—4t) _ 2 exp(—28)]. 

The characteristic equation is 27 —A—1=0. 

Un = [a + J/5)" — (1 = Ae (2",/5). 

From uy, and us, P = 5,Q 4. u, = 3/2 — 5(—1)"/6 + (—2)"/4 4+ 2"/12. 

The general solution is A+B2"”? exp(i3n/4)+C2"? exp(i52n/4). The initial values 
imply that A = 1/5,B = (/5/10) exp[i(z — #)] and C = (,/5/10) exp[i(z + @)]. 
This is Euler’s equation; setting x = expt produces d*z/dt* — 2dz/dt +z = expt, 
with complementary function (A + Bt)expt and particular integral t?(exp t)/2; 
y(x) = x+ [xInx(1 + Inx)]/2. 

After multiplication through by x? the coefficients are such that this is an 
exact equation. The resulting first-order equation, in standard form, needs an 
integrating factor (x — 2)?/x?. 

Given the boundary conditions, it is better to work with sinhx and sinh(1 — x) 
than with e+*; G(x,é) = —[sinh(1 — é) sinh x]/sinh1 for x < é and —[sinh(1 — 
x) sinh €]/sinh1 for x > é. 

Follow the method of subsection 15.2.5, but using general rather than specific 
functions. 

G(t,t) =0 5 t <T and tet) sin|ic(t —t)] for t >t. For a unit step input, 
x(t) = (1+ «?)-'(1 — ee cos xt — xe sin xt). Both transforms are equivalent to 
s[(s +1? + ae =1, 
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15.31 Use continuity and the step condition on 0G/0t at t = to to show that 
G(t,to.) =a {1 — exp[a(to — t)]} for 0 < to <t; 

x(t) = A(a — a)'{aq'[1 — exp(—at)] — «'[1 — exp(—at)]}. 

15.33 The LHS of the equation is exact for two stages of integration and then needs 
an integrating factor exp x; 2y d?y/dx? + 2y dy/dx + 2(dy/dx)’; 2y dy/dx + y? = 
d(y*)/dx + y?; y? = Aexp(—x) + Bx + C — (sin x — cos x)/2. 

15.35 Follow the method of subsection 15.2.6; u(x) = e- and v(x) satisfies v’ + 4v = 
sin 2x, for which a particular integral is (—xcos2x)/4. The general solution is 
y(x) = [Asin 2x + (B— 7X) cos 2xJe-", 

15.37. The equation is isobaric, with y of weight m, where m+ p— 2 = mn; v(x) 
satisfies xv" + xv’ = v". Set x = e' and v(x) = u(t), leading to u” = u" with 
u(0) = 0,u'(0) = 2. Multiply both sides by u’ to make the equation exact. 
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16 


Series solutions of ordinary 
differential equations 


In the previous chapter the solution of both homogeneous and non-homogeneous 
linear ordinary differential equations (ODEs) of order > 2 was discussed. In par- 
ticular we developed methods for solving some equations in which the coefficients 
were not constant but functions of the independent variable x. In each case we 
were able to write the solutions to such equations in terms of elementary func- 
tions, or as integrals. In general, however, the solutions of equations with variable 
coefficients cannot be written in this way, and we must consider alternative 
approaches. 

In this chapter we discuss a method for obtaining solutions to linear ODEs 
in the form of convergent series. Such series can be evaluated numerically, and 
those occurring most commonly are named and tabulated. There is in fact no 
distinct borderline between this and the previous chapter, since solutions in terms 
of elementary functions may equally well be written as convergent series (i.e. the 
relevant Taylor series). Indeed, it is partly because some series occur so frequently 
that they are given special names such as sin x, cos x or expx. 

Since we shall be concerned principally with second-order linear ODEs in this 
chapter, we begin with a discussion of these equations, and obtain some general 
results that will prove useful when we come to discuss series solutions. 


16.1 Second-order linear ordinary differential equations 


Any homogeneous second-order linear ODE can be written in the form 


y" + p(x)y' + q(x)y = 0, (16.1) 


where y’ = dy/dx and p(x) and q(x) are given functions of x. From the previous 
chapter, we recall that the most general form of the solution to (16.1) is 


y(x) = cry1 (x) + c2y2(x), (16.2) 
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where yi(x) and yo(x) are linearly independent solutions of (16.1), and cy and co 
are constants that are fixed by the boundary conditions (if supplied). 

A full discussion of the linear independence of sets of functions was given 
at the beginning of the previous chapter, but for just two functions y; and y2 
to be linearly independent we simply require that y2 is not a multiple of y1. 
Equivalently, y; and y2 must be such that the equation 


c1yi(X) + e2y2(x) = 0 
is only satisfied for cj = cp = 0. Therefore the linear independence of yi(x) and 
y2(x) can usually be deduced by inspection but in any case can always be verified 
by the evaluation of the Wronskian of the two solutions, 


W(x) = | oO | = yuh — vay. (16.3) 


a t 
1 2 


If W(x) #0 anywhere in a given interval then y; and y» are linearly independent 
in that interval. 

An alternative expression for W(x), of which we will make use later, may be 
derived by differentiating (16.3) with respect to x to give 


W' = yiys + ViV2 — YoY — Vay = Vis — ViV2. 
Since both y; and y» satisfy (16.1), we may substitute for y/ and y) to obtain 


W' = —yi(pys + ay2) + (py + ayi)y2 = P12 — yiv2) = —pW. 
Integrating, we find 
x 
W(x) = Cexp i-/ pw) du} ; (16.4) 


where C is a constant. We note further that in the special case p(x) = 0 we obtain 
W =constant. 


>The functions y; = sinx and y, = cosx are both solutions of the equation y" +y = 


0. Evaluate the Wronskian of these two solutions, and hence show that they are linearly 
independent. 


The Wronskian of y; and y2 is given by 


W =yiys — yoy} sin’ x — cos? x = —1. 


Since W # 0 the two solutions are linearly independent. We also note that y” + y = 0 is 
a special case of (16.1) with p(x) = 0. We therefore expect, from (16.4), that W will be a 
constant, as is indeed the case. < 


From the previous chapter we recall that, once we have obtained the general 
solution to the homogeneous second-order ODE (16.1) in the form (16.2), the 
general solution to the inhomogeneous equation 


y" + p(x)y’ + q(x)y = f(x) (16.5) 
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can be written as the sum of the solution to the homogeneous equation y,(x) 
(the complementary function) and any function yp(x) (the particular integral) that 
satisfies (16.5) and is linearly independent of y,(x). We have therefore 


V(x) = crys(x) + C2y2(x) + Yp(x). (16.6) 


General methods for obtaining y,, that are applicable to equations with variable 
coefficients, such as the variation of parameters or Green’s functions, were dis- 
cussed in the previous chapter. An alternative description of the Green’s function 
method for solving inhomogeneous equations is given in the next chapter. For the 
present, however, we will restrict our attention to the solutions of homogeneous 
ODEs in the form of convergent series. 


16.1.1 Ordinary and singular points of an ODE 


So far we have implicitly assumed that y(x) is a real function of a real variable 
x. However, this is not always the case, and in the remainder of this chapter we 
broaden our discussion by generalising to a complex function y(z) of a complex 
variable z. 

Let us therefore consider the second-order linear homogeneous ODE 


y" + plz)y’ + 4(z) = 0, (16.7) 


where now y’ = dy/dz; this is a straightforward generalisation of (16.1). A full 
discussion of complex functions and differentiation with respect to a complex 
variable z is given in chapter 24, but for the purposes of the present chapter we 
need not concern ourselves with many of the subtleties that exist. In particular, 
we may treat differentiation with respect to z in a way analogous to ordinary 
differentiation with respect to a real variable x. 

In (16.7), if, at some point z = zo, the functions p(z) and q(z) are finite and can 
be expressed as complex power series (see section 4.5), ie. 


ioe) ioe) 
p(z)= > palz— 20)", g(z) = D> an(z — 20)", 
n=0 n=0 
then p(z) and q(z) are said to be analytic at z = zo, and this point is called an 
ordinary point of the ODE. If, however, p(z) or q(z), or both, diverge at z = zo 
then it is called a singular point of the ODE. 

Even if an ODE is singular at a given point z = zo, it may still possess a 
non-singular (finite) solution at that point. In fact the necessary and sufficient 
condition’ for such a solution to exist is that (z — zo)p(z) and (z — zo)?q(z) are 
both analytic at z = zo. Singular points that have this property are called regular 


S See, for example, H. Jeffreys and B. S. Jeffreys, Methods of Mathematical Physics, 3rd edn (Cam- 
bridge: Cambridge University Press, 1966), p. 479. 
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singular points, whereas any singular point not satisfying both these criteria is 
termed an irregular or essential singularity. 


> Legendre’s equation has the form 
(1—2)y" — 2zy’ + 4(¢ + Ly =0, (16.8) 


where ¢ is a constant. Show that z = 0 is an ordinary point and z = +1 are regular singular 
points of this equation. 


Firstly, divide through by 1 — z? to put the equation into our standard form (16.7): 
2z , ef +1) 
1-2” y Lae 
Comparing this with (16.7), we identify p(z) and q(z) as 

(2) —2z —2z (2) (¢+1) ¢(¢ +1) 
P 1-2 (+z1—-z) 7% 1-2 ~ @+2\4—-2) 
By inspection, p(z) and q(z) are analytic at z = 0, which is therefore an ordinary point, 
but both diverge for z = +1, which are thus singular points. However, at z = 1 we see 
that both (z — 1)p(z) and (z — 1)?q(z) are analytic and hence z = 1 is a regular singular 
point. Similarly, at z = —1 both (z + 1)p(z) and (z + 1)*q(z) are analytic, and it too is a 
regular singular point. < 


=0. 


So far we have assumed that zo is finite. However, we may sometimes wish to 
determine the nature of the point |z| — oo. This may be achieved straightforwardly 
by substituting w = 1/z into the equation and investigating the behaviour at 
w=0. 


>» Show that Legendre’s equation has a regular singularity at |z| > ©. 


Letting w = 1/z, the derivatives with respect to z become 
dy dy dw 1 dy 2dy 
dz dwdz z> dw dw’ 


Py _ dw d (dy 2 dy 4d’y 3(o4v , ay 
=e (g)-- (- wh wth) = Gr oe or 


If we substitute these derivatives into Legendre’s equation (16.8) we obtain 


1 dy d’y 1 ,dy 
1-— 2—+w 2-— ¢(¢+1)y =0 
( a)" (23 ew) + wi + oe + Dy , 


which simplifies to give 


ay dy 
2/42 3 
w (we — 12 5 + 2w dw 
Dividing through by w?(w? — 1) to put the equation into standard form, and comparing 
with (16.7), we identify p(w) and q(w) as 
2w ; ¢(¢ +1) 
w2 — 1? ae w?(w? — 1)" 


At w = 0, p(w) is analytic but q(w) diverges, and so the point |z| — oo is a singular point 
of Legendre’s equation. However, since wp and w7q are both analytic at w = 0, |z| > oo 
is a regular singular point. < 


+¢(¢+1)y =0. 


p(w) = 
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Equation Regular Essential 
singularities singularities 
Hypergeometric 
z(1—z)y" + [c—(a+b+1)z]y’ —aby =0 0, 1, 00 — 
Legendre 
(1 —2*)y" —2zy'+¢(@+ ly =0 —1,1,0 _ 


Associated Legendre 


a 4 pi fp m 
(1 —27)y"” —2zy' + C2 a eer y= 


Chebyshev 

(1—2*)y"—zy'+wy=0 —1, 1,00 — 
Confluent hypergeometric 

zy" +(c—z)y'-—ay=0 0 fee) 
Bessel 

zy" + zy! + (22 —v*)y =0 0 foe) 
Laguerre 

zy"+(1—z)y’+vy =0 0 oe) 
Associated Laguerre 

zy" +(m+1—z)y'+(v—m)y =0 0 oe) 
Hermite 

y” —2zy'+2vy =0 — fea) 


Simple harmonic oscillator 
y"+o7y =0 — 0 


Table 16.1 Important second-order linear ODEs in the physical sciences and 
engineering. 


Table 16.1 lists the singular points of several second-order linear ODEs that 
play important roles in the analysis of many problems in physics and engineering. 
A full discussion of the solutions to each of the equations in table 16.1 and their 
properties is left until chapter 18. We now discuss the general methods by which 
series solutions may be obtained. 


16.2 Series solutions about an ordinary point 


If z = zo is an ordinary point of (16.7) then it may be shown that every solution 
y(z) of the equation is also analytic at z = zo. From now on we will take zo as the 
origin, Le. z) = 0. If this is not already the case, then a substitution Z = z — zo 
will make it so. Since every solution is analytic, y(z) can be represented by a 
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power series of the form (see section 24.11) 


i) = >o we" (16.9) 
n=0 
Moreover, it may be shown that such a power series converges for |z| < R, where 
R is the radius of convergence and is equal to the distance from z = 0 to the 
nearest singular point of the ODE (see chapter 24). At the radius of convergence, 
however, the series may or may not converge (as shown in section 4.5). 

Since every solution of (16.7) is analytic at an ordinary point, it is always 
possible to obtain two independent solutions (from which the general solution 
(16.2) can be constructed) of the form (16.9). The derivatives of y with respect to 
z are given by 


Yo =o naz"! = SO(n Ft Vansiz", (16.10) 
n=0 n=0 

y" = J" n(n Vagz”? = Sn + 2) + Vayyr2”. (16.11) 
n=0 n=0 


Note that, in each case, in the first equality the sum can still start at n = 0 since 
the first term in (16.10) and the first two terms in (16.11) are automatically zero. 
The second equality in each case is obtained by shifting the summation index 
so that the sum can be written in terms of coefficients of z”. By substituting 
(16.9)-(16.11) into the ODE (16.7), and requiring that the coefficients of each 
power of z sum to zero, we obtain a recurrence relation expressing each a, in 
terms of the previous a, (0<r<n-—1). 


> Find the series solutions, about z = 0, of 


y’+y=0. 


By inspection, z = 0 is an ordinary point of the equation, and so we may obtain two 
independent solutions by making the substitution y = )>*°9 a,z”. Using (16.9) and (16.11) 
we find 


Son aa 2)(n + 1)an422" a Sy yz" = 0, 


n=0 n=0 
which may be written as 


oe) 


Soll 2M F Vanya + ay]2" = 0. 


n=0 

For this equation to be satisfied we require that the coefficient of each power of z vanishes 
separately, and so we obtain the two-term recurrence relation 

an 


“(n+ 2)(n+l 


Using this relation, we can calculate, say, the even coefficients az, a4, ag and so on, for 


any = for n> 0. 
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a given do. Alternatively, starting with a;, we obtain the odd coefficients a3, as, etc. Two 
independent solutions of the ODE can be obtained by setting either a) = 0 or a; = 0. 
Firstly, if we set a, = 0 and choose ap = 1 then we obtain the solution 


2 74 ee (—1)" 7 
1 nh 
yi(z) ata > (On)! 
Secondly, if we set a9 = 0 and choose a; = 1 then we obtain a second, independent, solution 
Zz 2 = (-1) 2n+1 
yo(z) = 2 art 5 > On+ 1)! 


Recognising these two series as cosz and sinz, we can write the general solution as 
y(z) =c¢, cosz +c) sinz, 

where c; and cp are arbitrary constants that are fixed by boundary conditions (if supplied). 
We note that both solutions converge for all z, as might be expected since the ODE 
possesses no singular points (except |z| — 00). < 

Solving the above example was quite straightforward and the resulting series 
were easily recognised and written in closed form (i.e. in terms of elementary 
functions); this is not usually the case. Another simplifying feature of the previous 
example was that we obtained a two-term recurrence relation relating a,;. and 
dy, SO that the odd- and even-numbered coefficients were independent of one 
another. In general, the recurrence relation expresses a, in terms of any number 
of the previous a, (0< r<n-—1). 


> Find the series solutions, about z = 0, of 


By inspection, z = 0 is an ordinary point, and therefore we may find two independent 
solutions by substituting y = 57° 9 anz”. Using (16.10) and (16.11), and multiplying through 
by (1 —z)*, we find 


(1—2z +27) ie n(n — 1)dy,z"* —2 a2 a,z" = 0, 


n=0 n=0 


which leads to 


co) 


DS n(n — 1)a,z"-* — 2 5 n(n — 1a,z" + ss n(n — 1)a,z" — 2 x 4,2" = 0. 


n=0 n=0 n=0 n=0 


In order to write all these series in terms of the coefficients of z", we must shift the 
summation index in the first two sums, obtaining 


oe) 


SO(n t 2) + Vango2” —2 SO(n + Vnanyiz" + So? — n= 2)a,2" = 0, 


n=0 n=0 n=0 


which can be written as 


SO (at D+ 2)anp2 — 2ndnys + (N= 2)aq]2” = 0. 
n=0 
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By demanding that the coefficients of each power of z vanish separately, we obtain the 
three-term recurrence relation 


(n + 2)dn42 — 2ndn41 + (n — 2)a, =0 forn>0, 


which determines a, for n > 2 in terms of ao and a;. Three-term (or more) recurrence 
relations are a nuisance and, in general, can be difficult to solve. This particular recurrence 
relation, however, has two straightforward solutions. One solution is a, = do for all n, in 
which case (choosing ap = 1) we find 


1 
yi(z)=1tz24¢27474-05 , 
1-—z 
The other solution to the recurrence relation is a, = —2do, ay = dy and a, = 0 for n > 2, 
so that (again choosing dy) = 1) we obtain a polynomial solution to the ODE: 
yo(z) = 1-22 +2? = (1-2). 
The linear independence of y; and y> is obvious but can be checked by computing the 
Wronskian 


1 1 
W=y15 ay 2(1 
yy, Viy2 = 77-20 —2)] i—2 
Since W # 0, the two solutions y, and y, are indeed linearly independent. The general 
solution of the ODE is therefore 


(l—zy 3. 


mes +o(1—z). 

1-z 

We observe that y; (and hence the general solution) is singular at z = 1, which is the 
singular point of the ODE nearest to z = 0, but the polynomial solution, y., is valid for 
all finite z. < 


y(z) = 


The above example illustrates the possibility that, in some cases, we may find 
that the recurrence relation leads to a, = 0 for n > N, for one or both of the 
two solutions; we then obtain a polynomial solution to the equation. Polynomial 
solutions are discussed more fully in section 16.5, but one obvious property of 
such solutions is that they converge for all finite z. By contrast, as mentioned 
above, for solutions in the form of an infinite series the circle of convergence 
extends only as far as the singular point nearest to that about which the solution 
is being obtained. 


16.3 Series solutions about a regular singular point 


From table 16.1 we see that several of the most important second-order linear 
ODEs in physics and engineering have regular singular points in the finite complex 
plane. We must extend our discussion, therefore, to obtaining series solutions to 
ODEs about such points. In what follows we assume that the regular singular 
point about which the solution is required is at z = 0, since, as we have seen, if 
this is not already the case then a substitution of the form Z = z — zo will make 
it so. 
If z = 0 is a regular singular point of the equation 


y" + plz)y’ + q(z)y =0 
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then at least one of p(z) and q(z) is not analytic at z = 0, and in general we 
should not expect to find a power series solution of the form (16.9). We must 
therefore extend the method to include a more general form for the solution. In 
fact, it may be shown (Fuch’s theorem) that there exists at least one solution to 
the above equation, of the form 


y=? ys Anz", (16.12) 
n=0 
where the exponent o is a number that may be real or complex and where ay + 0 
(since, if it were otherwise, o could be redefined aso + 1 org +2 or --- so as to 
make ap # 0). Such a series is called a generalised power series or Frobenius series. 
As in the case of a simple power series solution, the radius of convergence of the 
Frobenius series is, in general, equal to the distance to the nearest singularity of 
the ODE. 
Since z = 0 is a regular singularity of the ODE, it follows that zp(z) and z?q(z) 
are analytic at z = 0, so that we may write 


sta) ste) = 
Z *q(z) =t(z)= Yt > 


where we have defined the analytic functions s(z) and t(z) for later convenience. 
The original ODE therefore becomes 
4 Oy 2), Ko, 
Zz Z 
Let us substitute the Frobenius series (16.12) into this equation. The derivatives 
of (16.12) with respect to z are given by 


y¥ => (nt o)azte, (16.13) 
n=0 

y" =SCnt ant —Vayz"*?, (16.14) 
n=0 


and we obtain 
s 


So(n + a)(n +0 —1)ayz"*** + s(z) Son + o)ayz"t?-? + t(z) oy an,z"*? =0. 


n=0 n=0 n=0 


Dividing this equation through by z°~?, we find 
ye [(n + o)(n+o6—1)+s(z)(n+ 6) + t(z)] ayz” = 0. (16.15) 
n=0 
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Setting z = 0, all terms in the sum with n > 0 vanish, implying that 
[a(o — 1) + s(O)o + t(0)]ao = 0, 
which, since we require ay # 0, yields the indicial equation 


o(a — 1) + s(0)o + t(0) =0. (16.16) 


This equation is a quadratic in o and in general has two roots, the nature of 
which determines the forms of possible series solutions. 

The two roots of the indicial equation, 0; and oo, are called the indices of 
the regular singular point. By substituting each of these roots into (16.15) in 
turn and requiring that the coefficients of each power of z vanish separately, we 
obtain a recurrence relation (for each root) expressing each a, as a function of 
the previous a, (0 < r < n—1). We will see that the larger root of the indicial 
equation always yields a solution to the ODE in the form of a Frobenius series 
(16.12). The form of the second solution depends, however, on the relationship 
between the two indices o; and o>. There are three possible general cases: (i) 
distinct roots not differing by an integer; (ii) repeated roots; (ili) distinct roots 
differing by an integer (not equal to zero). Below, we discuss each of these in turn. 

Before continuing, however, we note that, as was the case for solutions in 
the form of a simple power series, it is always worth investigating whether a 
Frobenius series found as a solution to a problem is summable in closed form 
or expressible in terms of known functions. We illustrate this point below, but 
the reader should avoid gaining the impression that this is always so or that, if 
one worked hard enough, a closed-form solution could always be found without 
using the series method. As mentioned earlier, this is not the case, and very often 
an infinite series solution is the best one can do. 


16.3.1 Distinct roots not differing by an integer 


If the roots of the indicial equation, o; and op, differ by an amount that is not 
an integer then the recurrence relations corresponding to each root lead to two 
linearly independent solutions of the ODE: 


ioe) ioe) 
yi(z) = 27! ye Anz", y2(z) = 2 a byz”, 

n=0 n=0 
with both solutions taking the form of a Frobenius series. The linear independence 
of these two solutions follows from the fact that y2/y; is not a constant since 
01 — 62 is not an integer. Because y; and yp are linearly independent, we may use 
them to construct the general solution y = c1y1 + c2yo. 

We also note that this case includes complex conjugate roots where o2 = oj, 

since o) — 062 = 0, — 0; = 2iImo, cannot be equal to a real integer. 
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> Find the power series solutions about z = 0 of 


4zy"+2y'+y=0. 


Dividing through by 4z to put the equation into standard form, we obtain 


Aine ty 

ar a 
and on comparing with (16.7) we identify p(z) = 1/(2z) and q(z) = 1/(4z). Clearly z =0 
is a singular point of (16.17), but since zp(z) = 1/2 and z*q(z) = 2/4 are finite there, it 
is a regular singular point. We therefore substitute the Frobenius series y = 27 0.9 dy” 
into (16.17). Using (16.13) and (16.14), we obtain 


0, (16.17) 


dn +a)(n+o—1)a,z"t?? + re dn + a)a,z"te-! + ce dX a,z"t? = 0, 
which, on dividing through by z’~*, gives 
Sy [(n+o)n+o—1)+4(n+o)+ 42] az" = 0. (16.18) 


n=0 
If we set z = 0 then all terms in the sum with n > 0 vanish, and we obtain the indicial 
equation 
a(¢ —1)+ 40 =0, 
which has roots o = 1/2 and o = 0. Since these roots do not differ by an integer, we 
expect to find two independent solutions to (16.17), in the form of Frobenius series. 


Demanding that the coefficients of z" vanish separately in (16.18), we obtain the 
recurrence relation 


(n+ oa)(n+o—1)a,+ 5(n +a)a,+ $n = 0. (16.19) 
If we choose the larger root, g = 1/2, of the indicial equation then (16.19) becomes 
24 = et 
(4n° + 2n)ay + dy_1 =0 = An Inn +1)" 
Setting do = 1, we find a, = (—1)"/(2n + 1)!, and so the solution to (16.17) is given by 


Jz (2) + (yz? +++ = sin \/z. 


To obtain the second solution we set o = 0 (the smaller root of the indicial equation) in 
(16.19), which gives 
An-1 


2 
(4n° — 2n)a, + dy) = 9 an 2n(2n — 1)" 


Setting ad) = 1 now gives a, = (—1)"/(2n)!, and so the second (independent) solution to 
(16.17) is 


yo(z) = 


5 1 EW cos 


4! 


541 


SERIES SOLUTIONS OF ORDINARY DIFFERENTIAL EQUATIONS 


We may check that y;(z) and y2(z) are indeed linearly independent by computing the 
Wronskian as follows: 


W = yiys — yoy 

; i er 1 

sin \/z (-5 sin ve) —cos \/z (a cos 7) 
= —ap (sin? Vz +008" \/z) = a 


Since W # 0, the solutions y,(z) and y2(z) are linearly independent. Hence, the general 
solution to (16.17) is given by 


z)=c,sin /z +) cos \/z. € 


16.3.2 Repeated root of the indicial equation 


If the indicial equation has a repeated root, so that o; = 02 =o, then obviously 
only one solution in the form of a Frobenius series (16.12) may be found as 
described above, i.e. 


z)=2" y Anz". 
n=0 


Methods for obtaining a second, linearly independent, solution are discussed in 
section 16.4. 


16.3.3 Distinct roots differing by an integer 


Whatever the roots of the indicial equation, the recurrence relation corresponding 
to the larger of the two always leads to a solution of the ODE. However, if the 
roots of the indicial equation differ by an integer then the recurrence relation 
corresponding to the smaller root may or may not lead to a second linearly 
independent solution, depending on the ODE under consideration. Note that for 
complex roots of the indicial equation, the ‘larger’ root is taken to be the one 
with the larger real part. 


> Find the power series solutions about z = 0 of 


z(z—1)y"+3zy +y =0. (16.20) 


Dividing through by z(z — 1) to put the equation into standard form, we obtain 


” 3 y 1 = 
yok Gai. sb da = i? =0, (16.21) 
and on comparing with (16.7) we identify p(z) = 3/(z — 1) and q(z) = 1/[z(z — 1)]. We 
immediately see that z = 0 is a singular point : (16.21), a since zp(z) = 3z/(z — 1) and 


z°q(z) = z/(z—1) are finite there, it is a regular singular point and we expect to find at least 
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one solution in the form of a Frobenius series. We therefore substitute y = 2° 77° 9 anz” 
into (16.21) and, using (16.13) and (16.14), we obtain 


Yon +a)(n+o—1)a,z"t?? + = 


n=0 n=0 


z(z— 1) art 
which, on dividing through by z7~’, gives 
~ 3 
Y [wtovnte n+ : (n+o)+ : | ae" =o. 
= z—1 z—l1 


Although we could use this expression to find the indicial equation and recurrence relations, 
the working is simpler if we now multiply through by z — 1 to give 


> [(z —1(n+ o)(n+o6—1)4+3z(n+ 6) +2] a,z”" = 0. (16.22) 


n=0 


If we set z = 0 then all terms in the sum with the exponent of z greater than zero vanish, 
and we obtain the indicial equation 


a(o — 1) =0, 
which has the roots o = 1 and o = 0. Since the roots differ by an integer (unity), it may not 
be possible to find two linearly independent solutions of (16.21) in the form of Frobenius 
series. We are guaranteed, however, to find one such solution corresponding to the larger 
root,o = 1. 


Demanding that the coefficients of z" vanish separately in (16.22), we obtain the 
recurrence relation 


(n —1+ o)(n —2+ O)Ay—-1 | (n a a)(n io = 1a, ote 3(n —1+ O)An—-1 +4), = 0, 
which can be simplified to give 
(n+o6—1)a, = (n+ 0)an-1. (16.23) 


On substituting ¢ = 1 into this expression, we obtain 


(* + ~) 
a= n-1, 
n 


and on setting dy) = 1 we find a, =n+ 1; so one solution to (16.21) is given by 


ylz) =z 50 (n+ 12” = 2(1 + 22 + 327 +--+) 
n=0 
Zz 
=, 16.24 
=: (16.24) 
If we attempt to find a second solution (corresponding to the smaller root of the indicial 
equation) by setting o = 0 in (16.23), we find 


n 
a= n=d Gn-1- 


But we require ap # 0, so a; is formally infinite and the method fails. We discuss how to 
find a second linearly independent solution in the next section. < 


One particular case is worth mentioning. If the point about which the solution 
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is required, i.e. z = 0, is in fact an ordinary point of the ODE rather than a 
regular singular point, then substitution of the Frobenius series (16.12) leads to 
an indicial equation with roots ¢ = 0 and o = 1. Although these roots differ by 
an integer (unity), the recurrence relations corresponding to the two roots yield 
two linearly independent power series solutions (one for each root), as expected 
from section 16.2. 


16.4 Obtaining a second solution 


Whilst attempting to construct solutions to an ODE in the form of Frobenius 
series about a regular singular point, we found in the previous section that when 
the indicial equation has a repeated root, or roots differing by an integer, we can 
(in general) find only one solution of this form. In order to construct the general 
solution to the ODE, however, we require two linearly independent solutions y; 
and y>. We now consider several methods for obtaining a second solution in this 
case. 


16.4.1 The Wronskian method 
If y; and y> are two linearly independent solutions of the standard equation 
y" + plz)y' + a(z)y =0 


then the Wronskian of these two solutions is given by W(z) = yiy5 — yoy}. 
Dividing the Wronskian by y? we obtain 


W_yy Vi V5 E (-)| d (2) 
os es + ee ‘ 
yim yy ve yt dz \y y2 dz \y 


which integrates to give 


7 W(u) 
yi(u) 


2 
i 
Now using the alternative expression for W(z) given in (16.4) with C = 1 (since 
we are not concerned with this normalising factor), we find 


pie) if set | pi) ao} du. (16.25) 
1 


Hence, given y;, we can in principle compute y». Note that the lower limits of 
integration have been omitted. If constant lower limits are included then they 
merely lead to a constant times the first solution. 


> Find a second solution to (16.21) using the Wronskian method. 


For the ODE (16.21) we have p(z) = 3/(z — 1), and from (16.24) we see that one solution 


yo(z) = yi(Z) 
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to (16.21) is y, = z/(1 — z)?. Substituting for p and y; in (16.25) we have 


4 (1= 4 u 3 


Z 4 
in Oe Caw exp [—3In(u —1)] du 


By calculating the Wronskian of y; and yp it is easily shown that, as expected, the two 
solutions are linearly independent. In fact, as the Wronskian has already been evaluated 
as W(u) = exp[—3 In(u — 1)], ie. W(z) = (z — 1)~3, no calculation is needed. < 


An alternative (but equivalent) method of finding a second solution is simply to 
assume that the second solution has the form y2(z) = u(z)y1(z) for some function 
u(z) to be determined (this method was discussed more fully in subsection 15.2.3). 
From (16.25), we see that the second solution derived from the Wronskian is 
indeed of this form. Substituting yo(z) = u(z)yi(z) into the ODE leads to a 
first-order ODE in which w’ is the dependent variable; this may then be solved. 


16.4.2 The derivative method 


The derivative method of finding a second solution begins with the derivation of 
a recurrence relation for the coefficients a, in a Frobenius series solution, as in the 
previous section. However, rather than putting o = o, in this recurrence relation 
to evaluate the first series solution, we now keep o as a variable parameter. This 
means that the computed a, are functions of o and the computed solution is now 
a function of z and a: 


y(z,0) = 2° > a,(a)z". (16.26) 
n=0 


Of course, if we put o = o; in this, we obtain immediately the first series solution, 
but for the moment we leave o as a parameter. 

For brevity let us denote the differential operator on the LHS of our standard 
ODE (16.7) by £, so that 


2 


d d 
L= ID + plz) +q(z), 


and examine the effect of £ on the series y(z,o) in (16.26). It is clear that the 
series Ly(z,o) will contain only a term in z’, since the recurrence relation defining 
the a,(c) is such that these coefficients vanish for higher powers of z. But the 
coefficient of z’ is simply the LHS of the indicial equation. Therefore, if the roots 
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of the indicial equation are o = o; and o = 02 then it follows that 
Ly(z,0) = ao(o — 01)(6 — 02)z”. (16.27) 


Therefore, as in the previous section, we see that for y(z,a) to be a solution of 
the ODE Ly = 0, o must equal oj or o2. For simplicity we shall set ap = 1 in the 
following discussion. 

Let us first consider the case in which the two roots of the indicial equation 
are equal, 1.e. 0. = a;. From (16.27) we then have 


Ly(z,6) = (6 —0)°2”. 
Differentiating this equation with respect to o we obtain 


0 


[Ly(z,o)] =(o — o1)°2° Inz + 2(6 — 0} )z", 


00 
which equals zero if o = o;. But since 0/00 and CL are operators that differentiate 
with respect to different variables, we can reverse their order, implying that 


00 


Hence, the function in square brackets, evaluated at o = o, and denoted by 


0 
£. Fac) =0 ata =9o}. 


one.o)| a (16.28) 


is also a solution of the original ODE Ly = 0, and is in fact the second linearly 
independent solution that we were looking for. 

The case in which the roots of the indicial equation differ by an integer is 
slightly more complicated but can be treated in a similar way. In (16.27), since £ 
differentiates with respect to z we may multiply (16.27) by any function of o, say 
0 — 0», and take this function inside the operator £ on the LHS to obtain 


L[(o — 02)y(z,0)] = (6 — o1)(o — 02)°2°. (16.29) 
Therefore the function 
l(o _ 62) y(z, O)e=0, 


is also a solution of the ODE Ly = 0. However, it can be proved’ that this 
function is a simple multiple of the first solution y(z,o,), showing that it is not 
linearly independent and that we must find another solution. To do this we 
differentiate (16.29) with respect to o and find 


F (£0 —o2)y(2, 0} = (6 — 02)'2" + 0 — oo — 02)2" 


+(¢ —o1)(6 — 02)°2" Inz, 


S For a fuller discussion see, for example, K. F. Riley, Mathematical Methods for the Physical Sciences 
(Cambridge: Cambridge University Press, 1974), pp. 158-9. 
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which is equal to zero if o = o>. As previously, since 0/do and CL are operators 
that differentiate with respect to different variables, we can reverse their order to 
obtain 


0 


a 


£{ S o-one.oy} =o at 0 =0), 

and so the function 

{= [(o — o2)y(z, on} (16.30) 
oo o=02 


is also a solution of the original ODE Ly = 0, and is in fact the second linearly 
independent solution. 


> Find a second solution to (16.21) using the derivative method. 


From (16.23) the recurrence relation (with o as a parameter) is given by 
(n+o6—1)a, = (n+ o0)ay-1. 


Setting ao = 1 we find that the coefficients have the particularly simple form a,(o) = 
(o +n)/o. We therefore consider the function 


oe) 


y(zZ,o) = 2? a a,(o)z" = z° ss Coen Zz” 


oO 
n=0 n=0 


The smaller root of the indicial equation for (16.21) is 7 = 0, and so from (16.30) a 
second, linearly independent, solution to the ODE is given by 


0 6 o . n 
{gg loveon} xt |: Vio +me 


o=0 


The derivative with respect to o is given by 


£ G so t+n)z"| =z? lnz S70 +n)z"+2° S Zz". 


n=0 n=0 n=0 


which on setting o = 0 gives the second solution 


y2(z) = Inz 3 nz" + > zn 
n=0 


n=0 


2 Inz+ 
~ (1=zy 1—z 


Zz 1 
iz (in +2 i). 


This second solution is the same as that obtained by the Wronskian method in the previous 
subsection except for the addition of some of the first solution. < 


16.4.3 Series form of the second solution 


Using any of the methods discussed above, we can find the general form of the 
second solution to the ODE. This form is most easily found, however, using the 
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derivative method. Let us first consider the case where the two solutions of the 
indicial equation are equal. In this case a second solution is given by (16.28), 
which may be written as 


_ [| éy(z,¢) 
y2(z) = | AG io 
1 - n 1 — da,(o) n 
= (Inz)z 2, onlor)e +z > | To 2 
= yi(z)Inz +27 >> yz", (16.31) 
n=1 


where b, = [da,(c)/do],=¢,. One could equally obtain the coefficients b, by direct 
substitution of the form (16.31) into the original ODE. 

In the case where the roots of the indicial equation differ by an integer (not 
equal to zero), then from (16.30) a second solution is given by 


y2(2) = lz [(o - crivie.oyl} ; 


ea 


>> FG - AG) ge 


o=07 


=Inz [« —0)z° 3 a,(a)z" 


n=0 


o=07 n=0 


But, as we mentioned in the previous section, [(o — 02)y(z,a)] at o = a2 Is just a 
multiple of the first solution y(z, o,). Therefore the second solution is of the form 


yo(z) = cy (z) nz +z” > byz", (16.32) 
n=0 


where c is a constant. In some cases, however, c might be zero, and so the second 
solution would not contain the term in Inz and could be written simply as a 
Frobenius series. Clearly this corresponds to the case in which the substitution 
of a Frobenius series into the original ODE yields two solutions automatically. 
In either case, the coefficients b, may also be found by direct substitution of the 
form (16.32) into the original ODE. 


16.5 Polynomial solutions 


We have seen that the evaluation of successive terms of a series solution to a 
differential equation is carried out by means of a recurrence relation. The form 
of the relation for a, depends upon n, the previous values of a, (r <n) and the 
parameters of the equation. It may happen, as a result of this, that for some 
value of n = N +1 the computed value ay, is zero and that all higher a, also 
vanish. If this is so, and the corresponding solution of the indicial equation o 
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is a positive integer or zero, then we are left with a finite polynomial of degree 
N’ =N-+0 asa solution of the ODE: 
N 
ia) = Soa" (16.33) 
n=0 
In many applications in theoretical physics (particularly in quantum mechanics) 
the termination of a potentially infinite series after a finite number of terms 
is of crucial importance in establishing physically acceptable descriptions and 
properties of systems. The condition under which such a termination occurs is 
therefore of considerable importance. 


> Find power series solutions about z = 0 of 
y" —2zy' + dy =0. (16.34) 


For what values of 2 does the equation possess a polynomial solution? Find such a solution 
for 4 =4. 


Clearly z = 0 is an ordinary point of (16.34) and so we look for solutions of the form 
VY => oy Anz". Substituting this into the ODE and multiplying through by z? we find 

x [n(n — 1) — 22?n + Az7Ja,z" = 0. 

n=0 
By demanding that the coefficients of each power of z vanish separately we derive the 
recurrence relation 

n(n — 1)a, — 2(n — 2)ay—2 + Ady-2 = 0, 

which may be rearranged to give 
_ 2(n—2)—4 
~ — n(n—1) 
The odd and even coefficients are therefore independent of one another, and two solutions 
to (16.34) may be derived. We either set a, = 0 and ap = 1 to obtain 


Q—-» forn>2. (16.35) 


Ay 


zt 26 


2: 
yi(z) = 1 as M4 — AZ —H4—AB- AE 


or set do = 0 and a; = 1 to obtain 


(16.36) 


2 2 z? 
y2(z) =z +Q-AT +(2-A6- AE + (2—AY(6— AO = Powis 


Now, from the recurrence relation (16.35) (or in this case from the expressions for y; 
and y2 themselves) we see that for the ODE to possess a polynomial solution we require 
A = 2(n — 2) for n > 2 or, more simply, 4 = 2n for n > 0, i.e. 2 must be an even positive 
integer. If 2 = 4 then from (16.36) the ODE has the polynomial solution 


42? 7 
TZ ee < 
A simpler method of obtaining finite polynomial solutions is to assume a 
solution of the form (16.33), where ay # 0. Instead of starting with the lowest 
power of z, as we have done up to now, this time we start by considering the 
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coefficient of the highest power 2% 


assumed form of solution. 


; such a power now exists because of our 


> By assuming a polynomial solution find the values of 4 in (16.34) for which such a solution 
exists. 


We assume a polynomial solution to (16.34) of the form y = pear a,z". Substituting this 
form into (16.34) we find 


N 
y [n(n — 1)dyz"? — 2znayz™! + Aanz"| se), 


n=0 


Now, instead of starting with the lowest power of z, we start with the highest. Thus, 
demanding that the coefficient of zY vanishes, we require —-2N + 2 = 0, ie. A = 2N, as we 
found in the previous example. By demanding that the coefficient of a general power of z 
is zero, the same recurrence relation as above may be derived and the solutions found. < 


16.6 Exercises 
16.1 Find two power series solutions about z = 0 of the differential equation 
(1—2”)y” —3zy’ +Ay =0. 
Deduce that the value of 4 for which the corresponding power series becomes an 
Nth-degree polynomial Uy(z) is N(N + 2). Construct U(z) and U3(z). 
16.2 Find solutions, as power series in z, of the equation 
4zy" + 2(1—z)y’-y =0. 
Identify one of the solutions and verify it by direct substitution. 
16.3 Find power series solutions in z of the differential equation 
zy" —2y' + 925y =0. 
Identify closed forms for the two series, calculate their Wronskian, and verify 
that they are linearly independent. Compare the Wronskian with that calculated 
from the differential equation. 
16.4 Change the independent variable in the equation 
d 
— a af =0 (*) 
dz 
from z to x = z —a, and find two independent series solutions, expanded about 
x = 0, of the resulting equation. Deduce that the general solution of (*) is 


™m! 


f(z,a) = A(z — we" ‘say Ore Im)! (z— a)", 
with A and B arbitrary constants. 
16.5 Investigate solutions of Legendre’s equation at one of its singular points as 


follows. 


(a) Verify that z = 1 is a regular singular point of Legendre’s equation and that 
the indicial equation for a series solution in powers of (z — 1) has roots 0 
and 3. 

(b) Obtain the corresponding recurrence relation and show that ¢ = 0 does not 
give a valid series solution. 


550 


16.6 EXERCISES 


16.6 


16.8 


16.9 


16.10 


(c) Determine the radius of convergence R of the o = 3 series and relate it to 
the positions of the singularities of Legendre’s equation. 


Verify that z = 0 is a regular singular point of the equation 
vy" —azy +(L+z)y =0, 


and that the indicial equation has roots 2 and 1/2. Show that the general solution 
is given by 


@. —1)"(n+1 y2n n 
He) = 602 > ~ ea 
7 u 


1)"2?"z n 
1/2 3/2 
+m (: oe va a0 te) 


Use the derivative method to obtain, as a second solution of Bessel’s equation 
for the case when v = 0, the following expression: 


ive) (—1)" n 1 Zz 2n 
Jz) he > (nl a (3) , 
given that the first solution is Jo(z), as specified by (18.79). 
Consider a series solution of the equation 
—2y'+yz=0  (*) 


about its regular singular point. 


(a) Show that its indicial equation has roots that differ by an integer but that 
the two roots nevertheless generate linearly independent solutions 


id 1)yPtl Anz2e1 
yi S305 


n+ 2n 
ate) Lae (-1) an 1)z 
n=0 
(b) Show that y,(z) is equal to 3ao(sinz — zcosz) by expanding the sinusoidal 
functions. Then, using the Wronskian method, find an expression for y2(z) in 
terms of sinusoids. You will need to write z? as (z/ sin z)(z sin z) and integrate 
by parts to evaluate the integral involved. 

(c) Confirm that the two solutions are linearly independent by showing that 
their Wronskian is equal to —z?, as would be expected from the form of (*). 


Find series solutions of the equation y” — 2zy’—2y = 0. Identify one of the series 
as yi(z) = exp z’ and verify this by direct substitution. By setting y2(z) = u(z)yi(z) 
and solving the resulting equation for u(z), find an explicit form for y.(z) and 


deduce that 
a ew dv = ee > n! (2x)! 
i “3Qn¢ II 


a d 
z(1 —2)55 +1 any 


Solve the equation 


y+ dy =0 
dz ay 
as follows. 


(a) Identify and classify its singular points and determine their indices. 
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(b) Find one series solution in powers of z. Give a formal expression for a 
second linearly independent solution. 

(c) Deduce the values of A for which there is a polynomial solution Py(z) of 
degree N. Evaluate the first four polynomials, normalised in such a way that 
Py(0) = 1. 


16.11 Find the general power series solution about z = 0 of the equation 


a 4 
2a 2 Ai Say oF + oe al 
16.12 Find the radius of convergence of a series solution about the origin for the 
equation (z? + az + b)y" + 2y = 0 in the following cases: 


(a)a=5, b=6; (b) a=5, b=7. 


Show that if a and b are real and 4b > a’, then the radius of convergence is 
always given by b!/?. 

16.13 For the equation y” + zy = 0, show that the origin becomes a regular singular 
point if the independent variable is changed from z to x = 1/z. Hence find a 
series solution of the form y;(z) = )\9 dnz~". By setting yo(z) = u(z)yi(z) and 
expanding the resulting expression for du/dz in powers of z~!, show that y2(z) 
has the asymptotic form 


yo(z) =c r+inz-$+0("2)| 5 


where c is an arbitrary constant. 
16.14 Prove that the Laguerre equation, 


ay dy 
Ss (ez 

"Tl i dz 
has polynomial solutions Ly(z) if 2 is a non-negative integer N, and determine 
the recurrence relationship for the polynomial coefficients. Hence show that an 
expression for Ly(z), normalised in such a way that Ly(0) = N}, is 


isthe popu yin? 


N —n)\(n!)? 
n=0 


pays, 


Evaluate L3(z) explicitly. 
16.15 The origin is an ordinary point of the Chebyshev equation, 


(1 == yy" = zy! phe my = 0, 
which therefore has series solutions of the form z° )7¢ 4,2" for ¢ =O ando = 1. 


(a) Find the recurrence relationships for the a, in the two cases and show that 
there exist polynomial solutions T,,,(z): 


(i) for o = 0, when m is an even integer, the polynomial having $(m +2) 
terms; 

(ii) for ¢ = 1, when m is an odd integer, the polynomial having $(m +1) 
terms. 


(b) T,,(z) is normalised so as to have T,,(1) = 1. Find explicit forms for T,,,(z) 
for m = 0,1, 2,3. 
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16.16 


16.13 


16.15 


(c) Show that the corresponding non-terminating series solutions S,,(z) have as 
their first few terms 


(2) =a (24+ 52+ geht), 


1 3 

S,(z) w(t a7 aie a), 
3 15 

S(z) wo (2 ae ae a), 


Sy(2) = (1 meet). 


2! 


Obtain the recurrence relations for the solution of Legendre’s equation (18.1) in 
inverse powers of z, i.e. set y(z) = )> a,z°-", with ag # 0. Deduce that, if 7 is an 
integer, then the series with o = 7 will terminate and hence converge for all z, 
whilst the series with o = —(¢ + 1) does not terminate and hence converges only 
for |z| > 1. 


16.7 Hints and answers 


Note that z = 0 is an ordinary point of the equation. 
For ¢ = 0, Gni2/d, = [n(n + 2) — A]/[(n + 1)(n + 2)] and, correspondingly, for 
o = 1, U2(z) = ao(1 — 42”) and U3(z) = ao(z — 223). 
o =0 and 35 dgm/ao = (—1)"/(2m)! and dgm/ag = (—1)"/(2m + 1)!, respectively. 
yi(z) = ag cos z? and y2(z) = do sinz?. The Wronskian is +3a3z” # 0. 
(b) duis /ay = [C0 + 1) — n(n + 1)]/[2(n + 1]. 
(c) R = 2, equal to the distance between z = 1 and the closest singularity at 
z=-1. 

(-1)"2?" 
[(o +2)(o+4)---(6 +2n)P 
The origin is an ordinary point. Determine the constant of integration by exam- 
ining the behaviour of the related functions for small x. 
ya(z) = (exp 22) J exp(—x°) dx. 
Repeated roots o = 2. 


A typical term in the series for y(¢,z) is 


(n+ 1)(—22)"2 ( 


y(z) = az? — 4az? + 6bz3 + +b [Inz+ g(n)) | ; 


n!} 4 
n=2 
where 
1 1 1 1 
a(n) n+1 on n—-l 2 2 


The transformed equation is xy” + 2y’+ y = 0; a, = (—1)"(n + 1)71(n!)-7a0; 
du/dz = A[yi(z)]-?. 

(a) (i) dango = [an(n? — n?)]/[(n + 2)(n + 1], 

(ii) dny2 = {an[(n + 1)? — m7 ]}/[(n + 3)(n + 2)]; (b) 1, 2, 227 — 1, 423 — 3z. 
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17 


Eigenfunction methods for 
differential equations 


In the previous three chapters we dealt with the solution of differential equations 
of order n by two methods. In one method, we found n independent solutions 
of the equation and then combined them, weighted with coefficients determined 
by the boundary conditions; in the other we found solutions in terms of series 
whose coefficients were related by (in general) an n-term recurrence relation and 
thence fixed by the boundary conditions. For both approaches the linearity of the 
equation was an important or essential factor in the utility of the method, and 
in this chapter our aim will be to exploit the superposition properties of linear 
differential equations even further. 

We will be concerned with the solution of equations of the inhomogeneous 
form 


Ly(x) = f(x), (17.1) 


where f(x) is a prescribed or general function and the boundary conditions to 
be satisfied by the solution y = y(x), for example at the limits x = a and x = b, 
are given. The expression £y(x) stands for a linear differential operator £ acting 
upon the function y(x). 

In general, unless f(x) is both known and simple, it will not be possible to 
find particular integrals of (17.1), even if complementary functions can be found 
that satisfy Cy = 0. The idea is therefore to exploit the linearity of £ by building 
up the required solution y(x) as a superposition, generally containing an infinite 
number of terms, of some set of functions {y;(x)} that each individually satisfy 
the boundary conditions. Clearly this brings in a quite considerable complication 
but since, within reason, we may select the set of functions to suit ourselves, we 
can obtain sizeable compensation for this complication. Indeed, if the set chosen 
is one containing functions that, when acted upon by CL, produce particularly 
simple results then we can ‘show a profit’ on the operation. In particular, if the 
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set consists of those functions y; for which 
Lyi(x) = Aivix), (17.2) 


where 4; is a constant (and which satisfy the boundary conditions), then a distinct 
advantage may be obtained from the manoeuvre because all the differentiation 
will have disappeared from (17.1). 

Equation (17.2) is clearly reminiscent of the equation satisfied by the eigenvec- 
tors x' of a linear operator A, namely 


Ax! = )ix', (17.3) 


where 4; is a constant and is called the eigenvalue associated with x'. By analogy, 
in the context of differential equations a function y,(x) satisfying (17.2) is called 
an eigenfunction of the operator £ (under the imposed boundary conditions) and 
4; is then called the eigenvalue associated with the eigenfunction y;(x). Clearly, 
the eigenfunctions y;(x) of £ are only determined up to an arbitrary scale factor 
by (17.2). 

Probably the most familiar equation of the form (17.2) is that which describes 
a simple harmonic oscillator, i.e. 


2 


Ly= 4 =w’y, where £L=—d*/dt?. (17.4) 


Imposing the boundary condition that the solution is periodic with period T, 
the eigenfunctions in this case are given by y,(t) = Ane, where w, = 2nn/T, 
n = 0,+1,+2,... and the A, are constants. The eigenvalues are we -_ wor = 
n?(2n/T)*. (Sometimes @, is referred to as the eigenvalue of this equation, but 
we will avoid such confusing terminology here.) 

We may discuss a somewhat wider class of differential equations by considering 
a slightly more general form of (17.2), namely 


Lyi(x) = Aip(x)yi(x), (17.5) 


where p(x) is a weight function. In many applications p(x) is unity for all x, in 
which case (17.2) is recovered; in general, though, it is a function determined by 
the choice of coordinate system used in describing a particular physical situation. 


The only requirement on p(x) is that it is real and does not change sign in the 
range a < x < b, so that it can, without loss of generality, be taken to be non- 
negative throughout; of course, p(x) must be the same function for all values of 
4;. A function y;(x) that satisfies (17.5) is called an eigenfunction of the operator 
£ with respect to the weight function p(x). 

This chapter will not cover methods used to determine the eigenfunctions of 
(17.2) or (17.5), since we have discussed those in previous chapters, but, rather, will 
use the properties of the eigenfunctions to solve inhomogeneous equations of the 
form (17.1). We shall see later that the sets of eigenfunctions y,(x) of a particular 
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class of operators called Hermitian operators (the operator in the simple harmonic 
oscillator equation is an example) have particularly useful properties and these 
will be studied in detail. It turns out that many of the interesting differential 
operators met within the physical sciences are Hermitian. Before continuing our 
discussion of the eigenfunctions of Hermitian operators, however, we will consider 
some properties of general sets of functions. 


17.1 Sets of functions 


In chapter 8 we discussed the definition of a vector space but concentrated on 
spaces of finite dimensionality. We consider now the infinite-dimensional space 
of all reasonably well-behaved functions f(x), g(x), h(x), .... on the interval 
a <x <b. That these functions form a linear vector space is shown by noting 
the following properties. The set is closed under 


(i) addition, which is commutative and associative, i.e. 


F(x) + g(x) = g(x) + f(x), 
[f(x) + g0x)] + h(x) = f(x) + [g(x) + hod], 


(ii) multiplication by a scalar, which is distributive and associative, i.e. 


ALf(x) + g(x] = Af(x) + Ag), 
A [uf (x)] = AW), 
(A+ wf (x) = Af (x) + uf). 
Furthermore, in such a space 


(iii) there exists a ‘null vector’ 0 such that f(x) +0 = f(x), 
(iv) multiplication by unity leaves any function unchanged, i.e. 1 x f(x) = f(x), 
(v) each function has an associated negative function —f(x) that is such that 


F(x) + [-f(x)] = 0. 


By analogy with finite-dimensional vector spaces we now introduce a set 
of linearly independent basis functions y,(x), n = 0,1,...,00, such that any 
‘reasonable’ function in the interval a < x < b (ie. it obeys the Dirichlet conditions 
discussed in chapter 12) can be expressed as the linear sum of these functions: 


ee) 


f(x) fa se CnYn(X). 


n=0 
Clearly if a different set of linearly independent basis functions u,(x) is chosen 
then the function can be expressed in terms of the new basis, 


f(x) = SS dyUn(X), 
n=0 
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where the d, are a different set of coefficients. In each case, provided the basis 
functions are linearly independent, the coefficients are unique. 
We may also define an inner product on our function space by 


b 
(flg) = / f*e(x)p(x) dx, (17.6) 


where p(x) is the weight function, which we require to be real and non-negative 
in the interval a < x < b. As mentioned above, p(x) is often unity for all x. Two 
functions are said to be orthogonal (with respect to the weight function p(x)) on 
the interval [a, b] if 


b 
(fle) = i f'ebop(x) dx = 0, (17.7) 


and the norm of a function is defined as 
1/2 


b 1/2 b 
i= (uy? =| | Ftsifloplo9ds| -| / {foots dx (178) 


It is also common practice to define a normalised function by f = f/\|f ||, which 
has unit norm. 

An infinite-dimensional vector space of functions, for which an inner product 
is defined, is called a Hilbert space. Using the concept of the inner product, we 
can choose a basis of linearly independent functions dn(x), n=0,1,2,... that are 
orthonormal, i.e. such that 


b 
(bild)) = | $id) p(e) dx = by. (179) 


If y,(x), n = 0,1,2,..., are a linearly independent, but not orthonormal, basis for 
the Hilbert space then an orthonormal set of basis functions dn may be produced 
(in a similar manner to that used in the construction of a set of orthogonal 
eigenvectors of an Hermitian matrix; see chapter 8) by the following procedure: 


do = Yo, 


b1 = V1 — bo(boly1), 
2 = yr — 1 (Pily2) — bo(doly2), 


Pn = Vn — bn-1 (dn—1|¥n) Tos ere dbo(dolyn)s 


It is straightforward to check that each ¢, is orthogonal to all its predecessors 
¢;, i = 0,1,2,...,n —1. This method is called Gram—Schmidt orthogonalisation. 
Clearly the functions ¢, form an orthogonal set, but in general they do not have 
unit norms. 
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> Starting from the linearly independent functions y,(x) = x", n = 0,1,..., construct three 
orthonormal functions over the range —1 < x < 1, assuming a weight function of unity. 


The first unnormalised function po is simply equal to the first of the original functions, i.e. 
go = 1. 
The normalisation is carried out by dividing by 
1/2 


(Golo)? = Ch 1x iu) aK/2, 


with the result that the first normalised function dbo is given by 


bo=- Sah 


The second unnormalised function is found by applying the above Gram-—Schmidt orthog- 
onalisation procedure, i.e. 


1 = y1 — bo(doly)- 
It can easily be shown that (boly1) = 0, and so ¢; = x. Normalising then gives 
1 1/2 = 
A 3 
b= di ( [ux nau) = 3x. 
The third unnormalised function is similarly given by 
2 = v2 — br (dily2) — bo(Poly2) 
=x7?—0-— i 


which, on normalising, gives 


h=h ae (uw? — 4)” in) = 1,/3(3x? — 1). 


By comparing the functions do, b1 and dy with the list in subsection 18.1.1, we see that 
this procedure has generated (multiples of) the first three Legendre polynomials. < 


If a function is expressed in terms of an orthonormal basis },(x) as 


a 


F(x) = So ena) (17.10) 


n=0 


then the coefficients c, are given by 


b 
cn = (Palf) -| PnlX)f (p(x) dx. (17.11) 
Note that this is true only if the basis is orthonormal. 
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17.1.1 Some useful inequalities 


Since for a Hilbert space (f|f) > 0, the inequalities discussed in subsection 8.1.3 
hold. The proofs are not repeated here, but the relationships are listed for 
completeness. 


(i) The Schwarz inequality states that 


l(flg)l < (If) (gle), (17.12) 


where the equality holds when f(x) is a scalar multiple of g(x), ie. when 
they are linearly dependent. 
(ii) The triangle inequality states that 


If +gll <llfll + Ilgl, (17.13) 


where again equality holds when f(x) is a scalar multiple of g(x). 
(iii) Bessel’s inequality requires the introduction of an orthonormal basis dn(X) 
so that any function f(x) can be written as 
f(x) = ye Cn@nlX), 


n=0 


where c, = (nlf). Bessel’s inequality then states that 
(fIf) = So lenl?. (17.14) 


The equality holds if the summation is over all the basis functions. If some 
values of n are omitted from the sum then the inequality results (unless, 
of course, the c, happen to be zero for all values of n omitted, in which 
case the equality remains). 


17.2 Adjoint, self-adjoint and Hermitian operators 


Having discussed general sets of functions, we now return to the discussion of 
eigenfunctions of linear operators. We begin by introducing the adjoint of an 
operator £, denoted by £1, which is defined by 


b b 
ii f* (x) [Lg(x)] dx = / [L*f(x)]" g(x) dx + boundary terms, 
i a (17.15) 


where the boundary terms are evaluated at the end-points of the interval [a, b]. 
Thus, for any given linear differential operator £, the adjoint operator Lt can be 
found by repeated integration by parts. 

An operator is said to be self-adjoint if £' = L. If, in addition, certain boundary 
conditions are met by the functions f and g on which a self-adjoint operator acts, 
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or by the operator itself, such that the boundary terms in (17.15) vanish, then the 
operator is said to be Hermitian over the interval a < x < b. Thus, in this case, 


b b 
i f° (x) [Le(x)] dx = 7 [Cf (x)]" g(x) dx. (17.16) 


A little careful study will reveal the similarity between the definition of an 
Hermitian operator and the definition of an Hermitian matrix given in chapter 8. 


> Show that the linear operator L = d?/dt? is self-adjoint, and determine the required 
boundary conditions for the operator to be Hermitian over the interval to to to + T. 


Substituting into the LHS of the definition of the adjoint operator (17.15) and integrating 


by parts gives 
to+T @ d to+T to+T df* d 
/ pean l|pZ / fds 
‘ dt? dt |, i dt dt 
0 0 


Integrating the second term on the RHS by parts once more yields 
to+T 2 d to+T df* to+T to+T gore 
| hr 
to u u to t to to t 


which, by comparison with (17.15), proves that £ is a self-adjoint operator. Moreover, 
from (17.16), we see that £ is an Hermitian operator over the required interval provided 


dg to+T df" to+T 
— = .< 
| 7 dt : 


to 0) 


We showed in chapter 8 that the eigenvalues of Hermitian matrices are real and 
that their eigenvectors can be chosen to be orthogonal. Similarly, the eigenvalues 
of Hermitian operators are real and their eigenfunctions can be chosen to be 
orthogonal (we will prove these properties in the following section). Hermitian 
operators (or matrices) are often used in the formulation of quantum mechanics. 
The eigenvalues then give the possible measured values of an observable quantity 
such as energy or angular momentum, and the physical requirement that such 
quantities must be real is ensured by the reality of these eigenvalues. Furthermore, 
the infinite set of eigenfunctions of an Hermitian operator form a complete basis 
set over the relevant interval, so that it is possible to expand any function y(x) 
obeying the appropriate conditions in an eigenfunction series over this interval: 


y(x) = S> caval), (17.17) 
n=0 
where the choice of suitable values for the c, will make the sum arbitrarily close 
to y(x)8 These useful properties provide the motivation for a detailed study of 
Hermitian operators. 


S The proof of the completeness of the eigenfunctions of an Hermitian operator is beyond the scope 
of this book. The reader should refer, for example, to R. Courant and D. Hilbert, Methods of 
Mathematical Physics (New York: Interscience, 1953). 
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17.3 Properties of Hermitian operators 


We now provide proofs of some of the useful properties of Hermitian operators. 
Again much of the analysis is similar to that for Hermitian matrices in chapter 8, 
although the present section stands alone. (Here, and throughout the remainder 
of this chapter, we will write out inner products in full. We note, however, that 
the inner product notation often provides a neat form in which to express results.) 


17.3.1 Reality of the eigenvalues 


Consider an Hermitian operator for which (17.5) is satisfied by at least two 
eigenfunctions y;(x) and y,(x), which have corresponding eigenvalues A; and 4,, 
so that 


Lyi = Aip(X) Vis (17.18) 
Lyj = Ajp(x)¥;, (17.19) 


where we have allowed for the presence of a weight function p(x). Multiplying 
(17.18) by y; and (17.19) by y; and then integrating gives 


b b 
| yjLyidx = af yj vip Ax, (17.20) 
a a 


b b 
| yiLyj ax =i; [ Vi yjp ax. (17.21) 
a a 


Remembering that we have required p(x) to be real, the complex conjugate of 
(17.20) becomes 


b b 
[vier ax =% [vive de, (17.22) 


and using the definition of an Hermitian operator (17.16) it follows that the LHS 
of (17.22) is equal to the LHS of (17.21). Thus 


b 
(Aj — a) f yj yjp ax =0. (17.23) 


If i = j then A; = 2; (since die y; yip dx # 0), which is a statement that the 
eigenvalue /; is real. 


17.3.2 Orthogonality and normalisation of the eigenfunctions 


From (17.23), it is immediately apparent that two eigenfunctions y; and y; that 
correspond to different eigenvalues, i.e. such that A; # /;, satisfy 


b 
/ y; yjp dx = 0, (17.24) 
a 
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which is a statement of the orthogonality of y; and y;. 

If one (or more) of the eigenvalues is degenerate, however, we have different 
eigenfunctions corresponding to the same eigenvalue, and the proof of orthogo- 
nality is not so straightforward. Nevertheless, an orthogonal set of eigenfunctions 
may be constructed using the Gram—Schmidt orthogonalisation method mentioned 
earlier in this chapter and used in chapter 8 to construct a set of orthogonal 
eigenvectors of an Hermitian matrix. We repeat the analysis here for complete- 
ness. 

Suppose, for the sake of our proof, that A» is k-fold degenerate, i.e. 


Ly, =Aopy; fori=0,1,...,k —1, (17.25) 


but that Ao is different from any of 2, 2441, etc. Then any linear combination of 
these y; is also an eigenfunction with eigenvalue A since 


k-1 k-1 k-1 
Lz= LS ci = ys; oly; = = ciAopy; = Agpz. (17.26) 
i=0 i=0 i=0 


If the y; defined in (17.25) are not already mutually orthogonal then consider 
the new eigenfunctions z; constructed by the following procedure, in which each 
of the new functions z; is to be normalised, to give 2;, before proceeding to the 
construction of the next one (the normalisation can be carried out by dividing 
the eigenfunction z; by ( be zi zip dx)'/?): 


20 = Yo, 


b 
21> yi— (% f Ziyipd) . 
a 
b b 
22 = y2—- (af 21 y2p ax) = (2 / 2020 ix) y 
a a 


b b 
Ze-1 = Ye-1 — (a | 24-2Vk 7) Sa (2 f Zin-apdx). 
a a 


Each of the integrals is just a number and thus each new function z; is, as can be 
shown from (17.26), an eigenvector of £ with eigenvalue Ap. It is straightforward 
to check that each z; is orthogonal to all its predecessors. Thus, by this explicit 
construction we have shown that an orthogonal set of eigenfunctions of an 
Hermitian operator £ can be obtained. Clearly the orthogonal set obtained, z;, is 
not unique. 

In general, since £ is linear, the normalisation of its eigenfunctions y;(x) is 
arbitrary. It is often convenient, however, to work in terms of the normalised 


AA 


eigenfunctions };(x), so that ais 3; Vio dx = 1. These therefore form an orthonormal 
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set and we can write 
b 
i 35 Dip dx = dij, (17.27) 
a 


which is valid for all pairs of values i, j. 


17.3.3, Completeness of the eigenfunctions 


As noted earlier, the eigenfunctions of an Hermitian operator may be shown 
to form a complete basis set over the relevant interval. One may thus expand 
any (reasonable) function y(x) obeying appropriate boundary conditions in an 
eigenfunction series over the interval, as in (17.17). Working in terms of the 
normalised eigenfunctions },(x), we may thus write 


b 
f= ful) SC2Vfe)ple) de 


b 
=f feVpt2) sncoyilerae 
Since this is true for any f(x), we must have that 


(2) 9- In(x) 97 (z) = 6(x — 2). (17.28) 


This is called the completeness or closure property of the eigenfunctions. It defines 
a complete set. If the spectrum of eigenvalues of £ is anywhere continuous then 
the eigenfunction y,(x) must be treated as y(n, x) and an integration carried out 
over n. 

We also note that the RHS of (17.28) is a 6-function and so is only non-zero 
when z = x; thus p(z) on the LHS can be replaced by p(x) if required, i.e. 


(2) D> Inl)P4(2) = ple) > ul H4(2). (17.29) 


17.3.4 Construction of real eigenfunctions 
Recall that the eigenfunction y; satisfies 
LYi = 4ipyi (17.30) 
and that the complex conjugate of this gives 
Ly; =4; py; = py; (17.31) 


where the last equality follows because the eigenvalues are real, ie. A; = 2;. 
Thus, y; and y; are eigenfunctions corresponding to the same eigenvalue and 
hence, because of the linearity of £, at least one of y; + y; and i(y; — y;), which 
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are both real, is a non-zero eigenfunction corresponding to that eigenvalue. It 
follows that the eigenfunctions can always be made real by taking suitable linear 
combinations, though taking such linear combinations will only be necessary in 
cases where a particular 4 is degenerate, 1.e. corresponds to more than one linearly 
independent eigenfunction. 


17.4 Sturm-—Liouville equations 


One of the most important applications of our discussion of Hermitian operators 
is to the study of Sturm—Liouville equations, which take the general form 


Wy ; 
dx2 | 


p(x) (17.32) 


ee + q(x)y + Ap(x)y =0, where r(x) = ants) 

dx dx 
and p, q and r are real functions of x.$ A variational approach to the Sturm— 
Liouville equation, which is useful in estimating the eigenvalues 4 for a given set 
of boundary conditions on y, is discussed in chapter 22. For now, however, we 
concentrate on demonstrating that solutions of the Sturm—Liouville equation that 
satisfy appropriate boundary conditions are the eigenfunctions of an Hermitian 
operator. 


It is clear that (17.32) can be written 


‘i d 
Ly =/p(x)y, where £L= POX) +r(x) re + q(x)}. (17.33) 


Using the condition that r(x) = p’(x), it will be seen that the general Sturm— 
Liouville equation (17.32) can also be rewritten as 


(py) + ay + py = 9, (17.34) 


where primes denote differentiation with respect to x. Using (17.33) this may also 
be written Ly = —(py’)' — gy = Apy, which defines a more useful form for the 
Sturm-—Liouville linear operator, namely 


pak F (oov-2) ie at| (17.35) 


17.4.1 Hermitian nature of the Sturm-Liouville operator 


As we now show, the linear operator of the Sturm—Liouville equation (17.35) is 
self-adjoint. Moreover, the operator is Hermitian over the range [a,b] provided 


S We note that sign conventions vary in this expression for the general Sturm—Liouville equation; 
some authors use —Ap(x)y on the LHS of (17.32). 
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certain boundary conditions are met, namely that any two eigenfunctions y; and 
y; of (17.33) must satisfy 


[yipyi],_, = [yipyi],_, for all ij. (17.36) 


Rearranging (17.36), we can write 

x=b 

Ly ry] =0 (17.37) 

x=a 
as an equivalent statement of the required boundary conditions. These boundary 
conditions are in fact not too restrictive and are met, for instance, by the sets 
y(a) = y(b) = 0; y(a) = y'(b) = 0; p(a) = p(b) = 0 and by many other sets. It 
is important to note that in order to satisfy (17.36) and (17.37) one boundary 
condition must be specified at each end of the range. 


> Prove that the Sturm—Liouville operator is Hermitian over the range [a,b] and under the 
boundary conditions (17.37). 


Putting the Sturm—Liouville form Ly = —(py’)’ — gy into the definition (17.16) of an 
Hermitian operator, the LHS may be written as a sum of two terms, i.e. 


b 


b b 
- / [vi (py) +yrayi] dx = — / y; (py;)' dx — i Vi qyj ax. 
The first term may be integrated by parts to give 
b b 
- E ry + / (y;)'py; ax. 
The boundary-value term in this is zero because of the boundary conditions, and so 
integrating by parts again yields 


b 


b 
loi V0») = / ((y;)'p)'yj dx. 


Again, the boundary-value term is zero, leaving us with 


a 


b 


b 
- / [yi (py;)' + ypayj] dx = — / [y(pO7yY + yjay;] ax, 


a 


which proves that the Sturm—Liouville operator is Hermitian over the prescribed interval. < 


It is also worth noting that, since p(a) = p(b) = 0 is a valid set of boundary 
conditions, many Sturm—Liouville equations possess a ‘natural’ interval [a, b] over 
which the corresponding differential operator £ is Hermitian irrespective of the 
boundary conditions satisfied by its eigenfunctions at x = a and x = b (the only 
requirement being that they are regular at these end-points). 


17.4.2 Transforming an equation into Sturm-Liouville form 


Many of the second-order differential equations encountered in physical problems 
are examples of the Sturm—Liouville equation (17.34). Moreover, any second-order 
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Equation D(x) q(x) 2 p(X) 
Hypergeometric xe(1 — xyttboetl 0 —ab xT (LP ix) atb-e 
Legendre 1—x? 0 U(¢ +1) 1 
Associated Legendre 1 x2 —m?/(1—x?2)  ¢(¢ +1) 1 
Chebyshev (1—x?)!/2 0 v? (1—x2)7!/2 
Confluent hypergeometric xe* 0 —a xe te 
Bessel” x —v?/x a x 
Laguerre xe—* 0 v e~* 
Associated Laguerre xt ox 0 v xe 
Hermite ew 0 2v e~ 
Simple harmonic 1 0 wo? 1 


Table 17.1 The Sturm—Liouville form (17.34) for important ODEs in the 
physical sciences and engineering. The asterisk denotes that, for Bessel’s equa- 
tion, a change of variable x — x/a is required to give the conventional 
normalisation used here, but is not needed for the transformation into Sturm— 
Liouville form. 


differential equation of the form 
P(x)y" + rOdy’ + aQdy + Ap(x)y = 0 (17.38) 


can be converted into Sturm—Liouville form by multiplying through by a suitable 
integrating factor, which is given by 


F(x) = exp {| no) ay) au (17.39) 
P(u) 
It is easily verified that (17.38) then takes the Sturm—Liouville form, 


[F(x)p(x)y'] + F(x)q(x)y + AF(x)p(x)y = 0, (17.40) 


with a different, but still non-negative, weight function F(x)p(x). Table 17.1 
summarises the Sturm—Liouville form (17.34) for several of the equations listed 
in table 16.1. These forms can be determined using (17.39), as illustrated in the 
following example. 


> Put the following equations into Sturm—Liouville (SL) form: 
(i) (1—x?)y”—xy’+wy=0 (Chebyshev equation); 


(ii) xy” +(1—x)y'+vy =0 (Laguerre equation) ; 
(iii) oy” —2xy’+2vy =0 (Hermite equation ). 


(i) From (17.39), the required integrating factor is 


F(x) = exp (f i — in) exp [ 3 In(d x’)] (1—x?)7!?, 


Thus, the Chebyshev equation becomes 
CaaS ey td sa) (ay) hielo) ey SO 
which is in SL form with p(x) = (1 — x7)!?, q(x) = 0, p(x) = (1 — x7)"!? and 2 = v?. 
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(ii) From (17.39), the required integrating factor is 


F(x) = exp (/ —1 in) = exp(—x). 


Thus, the Laguerre equation becomes 


xe *y" + (1—x)e“*y’ + ve“*y = (xe“y’)' + ve*y = 0, 


which is in SL form with p(x) = xe~*, q(x) = 0, p(x) =e* and 2=y. 


(iii) From (17.39), the required integrating factor is 


F(x) = exp Gs —2u au) = exp(—x°). 


Thus, the Hermite equation becomes 
ey" —2xe y! 4 2ve y = (e y')' + ve y =0, 
which is in SL form with p(x) = ee, q(x) = 0, p(x) = e™ and 1 =2v. < 


From the p(x) entries in table 17.1, we may read off the natural interval over 
which the corresponding Sturm—Liouville operator (17.35) is Hermitian; in each 
case this is given by [a,b], where p(a) = p(b) = 0. Thus, the natural interval 
for the Legendre equation, the associated Legendre equation and the Chebyshev 
equation is [—1,1]; for the Laguerre and associated Laguerre equations the 
interval is [0,00]; and for the Hermite equation it is [—oo, 0]. In addition, from 
(17.37), one sees that for the simple harmonic equation one requires only that 
[a,b] = [xo, x0 + 22]. We also note that, as required, the weight function in each 
case is finite and non-negative over the natural interval. Occasionally, a little more 
care is required when determining the conditions for a Sturm—Liouville operator 
of the form (17.35) to be Hermitian over some natural interval, as is illustrated 
in the following example. 


> Express the hypergeometric equation, 
x(1—x)y"” + [e—-(a+b+1)x]y’ —aby =0, 


in Sturm—Liouville form. Hence determine the natural interval over which the resulting 
Sturm—Liouville operator is Hermitian and the corresponding conditions that one must im- 
pose on the parameters a, b and c. 


As usual for an equation not already in SL form, we first determine the appropriate 
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integrating factor. This is given, as in equation (17.39), by 


fuNeeai 
du 
u(1 —u) 


F(x) = exp 


— 
= exp ————_ du 


[ f*fe-1 c-1 a+b-1 
=exp| [ (+ usd —u au) | 


=exp[(a+b—c)In(1—x)+(c—1)Inx] 


= x1 —_ x)aree, 


When the equation is multiplied through by F(x) it takes the form 


[ x(1 = x)erboet ly ] f abx*'(1 x)they 0. 


Now, for the corresponding Sturm—Liouville operator to be Hermitian, the conditions 
to be imposed are as follows. 


(i) The boundary condition (17.37); if c > 0 and a+b—c+1 > 0, this is satisfied 
automatically for 0 < x < 1, which is thus the natural interval in this case. 

(ii) The weight function x°—!(1 — x)**>-* must be finite and not change sign in the 
interval 0 < x < 1. This means that both exponents in it must be positive, Le. 
c—1>0Oanda+b—c>0. 


Putting together the conditions on the parameters gives the double inequality a+b >c> 
1.< 


Finally, we consider Bessel’s equation, 


x7y" xy! (x? v-)y =0, 


which may be converted into Sturm—Liouville form, but only in a somewhat 
unorthodox fashion. It is conventional first to divide the Bessel equation by x 
and then to change variables to x = x/«. In this case, it becomes 


2 
xy" (ax) + y!(ax) — = (2%) + o%y(ak) = 0, (17.41) 


where a prime now indicates differentiation with respect to x. Dropping the bars 
on the independent variable, we thus have 


2 
[xy"(ax)]’ — —y(ax) +o°xy(ax) = 0, (17.42) 
which is in SL form with p(x) = x, q(x) = —v?/x, p(x) = x and 2 = a. It 


should be noted, however, that in this case the eigenvalue (actually its square 
root) appears in the argument of the dependent variable. 
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17.5 Superposition of eigenfunctions: Green’s functions 


We have already seen that if 
Ly,(x) = AnX)Vn(X), (17.43) 


where £ is an Hermitian operator, then the eigenvalues 2, are real and the 
eigenfunctions y,(x) are orthogonal (or can be made so). Let us assume that we 
know the eigenfunctions y,(x) of £ that individually satisfy (17.43) and some 
imposed boundary conditions (for which £ is Hermitian). 

Now let us suppose we wish to solve the inhomogeneous differential equation 


Ly(x) = f(x), (17.44) 


subject to the same boundary conditions. Since the eigenfunctions of £ form a 
complete set, the full solution, y(x), to (17.44) may be written as a superposition 
of eigenfunctions, i.e. 


y(x) = So enynlx), (17.45) 
n=0 


for some choice of the constants c,. Making full use of the linearity of £, we have 


f(x) = Lyx) =L (>: cont) = So enLynlx) = S > endnp(x)yn(x). 


n=0 n=0 n=0 (17.46) 


Multiplying the first and last terms of (17.46) by y; and integrating, we obtain 


i yleted =o i CyAnys(2)ynl2)p(2) dz (17.47) 
n=0 
where we have used z as the integration variable for later convenience. Finally, 
using the orthogonality condition (17.27), we see that the integrals on the RHS 
are zero unless n = j, and so obtain 
b : 
: d 
oq = 1 Se Yale) de yore 
An fi YnlZ)Yn(z)p(z) dz 
Thus, if we can find all the eigenfunctions of a differential operator then (17.48) 
can be used to find the weighting coefficients for the superposition, to give as the 
full solution 


21 Sf yet 
(x) = eae eOHOL ES (X). (17.49) 
I ER enon 


If we work with normalised eigenfunctions },(x), so that 
b 

[serie yds = 1 for atin 
a 
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and we assume that we may interchange the order of summation and integration, 
then (17.49) can be written as 


b « 1 
ye) = | p> [phoonca| Vid. 


n=0 


The quantity in braces, which is a function of x and z only, is usually written 
G(x, z), and is the Green’s function for the problem. With this notation, 


b 
v(x) = i Glx,2)f (dz, (17.50) 
where 
Glx,2) = 7 Fula) Pn (2). (17.51) 
n=0 “" 


We note that G(x,z) is determined entirely by the boundary conditions and the 
eigenfunctions },, and hence by C itself, and that f(z) depends purely on the 
RHS of the inhomogeneous equation (17.44). Thus, for a given £ and boundary 
conditions we can establish, once and for all, a function G(x, z) that will enable 
us to solve the inhomogeneous equation for any RHS. From (17.51) we also note 
that 


G(x, z) = G"(z,x). (17.52) 


We have already met the Green’s function in the solution of second-order dif- 
ferential equations in chapter 15, as the function that satisfies the equation 
L[G(x,z)] = 6(x —z) (and the boundary conditions). The formulation given 
above is an alternative, though equivalent, one. 


> Find an appropriate Green’s function for the equation 


y’ +4y =f), 
with boundary conditions y(0) = y(z) = 0. Hence, solve for (i) f(x) = sin2x and (ii) 
f(x) = x/2. 


One approach to solving this problem is to use the methods of chapter 15 and find 
a complementary function and particular integral. However, in order to illustrate the 
techniques developed in the present chapter we will use the superposition of eigenfunctions, 
which, as may easily be checked, produces the same solution. 

The operator on the LHS of this equation is already Hermitian under the given boundary 
conditions, and so we seek its eigenfunctions. These satisfy the equation 


y+ qy =4y. 


This equation has the familiar solution 


v(x) = Asin i=) x+Bcos (4-2) x. 
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Now, the boundary conditions require that B = 0 and sin ( / ; —/)x=0, and so 


f-A=n, where n = 0,+1,+2,.... 


Therefore, the independent eigenfunctions that satisfy the boundary conditions are 


YnlX) = A, sin nx, 


where n is any non-negative integer, and the corresponding eigenvalues are 1, = ; —n., 


The normalisation condition further requires 


7 nV 12 
| A sin? nx dx = 1 => A, = (=) . 
0 T 


Comparison with (17.51) shows that the appropriate Green’s function is therefore given 
by 
2 GS sinnx sin nz 
G(x,z) = = —— 
(x, z) ; dX ii 


Case (i). Using (17.50), the solution with f(x) = sin2x is given by 


n[® a ; o 5 
2 sinnxsinnz \ . 2 sin nx : F 
y(x) == y ———_ ] sin2z dz = — y sin nz sin 2z dz. 
Tt Jo z—-wv Tt —w Jo 


n=0 a 


n=0 4 


Now the integral is zero unless n = 2, in which case it is 


ie 
[ si ZaZ 5) 


= 2sin2xn 4 Sas 
WOOT T5/42. ‘15 
is the full solution for f(x) = sin2x. This is, of course, exactly the solution found by using 
the methods of chapter 15. 
Case (ii). The solution with f(x) = x/2 is given by 


Thus 


"24 sinnxsinnz \ z 1 sinnx [*. 
yoo) = f = ——__] saz =- _/ zsinnz dz. 
6 ae ¢-—w 2 Der 0 
The integral may be evaluated by integrating by parts. For n # 0, 
1 
ee ZCOSNZ % 
i zsinnz dz | + | pele dz 
0 n 0 n 
0 
—1 cosnt sin nz 
eae ec 
n 


T 


n 
ewe 


n 


0 


For n = 0 the integral is zero, and thus 


, ye! sin nx ; 

v(x) 2 ae) 
is the full solution for f(x) = x/2. Using the methods of subsection 15.1.2, the solution 
is found to be y(x) = 2x — 2msin(x/2), which may be shown to be equal to the above 
solution by expanding 2x — 27 sin(x/2) as a Fourier sine series. < 
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17.6 A useful generalisation 


Sometimes we encounter inhomogeneous equations of a form slightly more gen- 
eral than (17.1), given by 


Ly(x) — up(x)yx) = fx) (17.53) 


for some Hermitian operator £, with y subject to the appropriate boundary 
conditions and y a given (i.e. fixed) constant. To solve this equation we expand 
y(x) and f(x) in terms of the eigenfunctions y,(x) of the operator £, which satisfy 


Lynx) = Anp(x)Yn(X). 


Working in terms of the normalised eigenfunctions },,(x), we first expand f(x) 
as follows: 


f) =37 940 i H5(2)f(e)ple) dz 


n=0 
-[ ple) > dace) fle) de. (17.54) 
n=0 
Using (17.29) this becomes 


b oO 
fc)= f peo) Yo Joboiteite de 


=H) 60) i Pee)f (2) dz. (17.55) 


n=0 


Next, we expand y(x) as y = )77°.9 CnJn(x) and seek the coefficients c,. Substituting 
this and (17.55) into (17.53) we have 


b 
As) lin OECD Be [ sere: 


n=0 


from which we find that 
b as 


Ja In(2)F 4 
Cn pes = 5 fa aE) ta ve = 


n=0 
Hence the solution of (17.53) is given by 


ee) 


oO 4 b 
y= Denti) =o Pf srcenpteyds = [ 5 LAE oye, 


n=0 n=0 “" a n=0 t 2 


From this we may identify the Green’s function 


G(x,z) = 3 ae, 
n=0 fn 
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We note that if w = Ay, Le. if « equals one of the eigenvalues of L, then G(x, z) 
becomes infinite and this method runs into difficulty. No solution then exists 
unless the RHS of (17.53) satisfies the relation 


b 
/ OOO) dx =0. 


If the spectrum of eigenvalues of the operator £ is anywhere continuous, 
the orthonormality and closure relationships of the normalised eigenfunctions 
become 


b 
/ IOP m(x)o(x) dx = 4(n — mi), 


| * $h(2\Dn) 00) dn = 5(x— 2). 


Repeating the above analysis we then find that the Green’s function is given by 


G(x,z) = if Inn) ty 
0 4n— 


17.7 Exercises 


17.1 By considering (h|h), where h = f + Ag with 2 real, prove that, for two functions 
f and g, 
(fif (gle) = gl(fle) + (gl fP. 


The function y(x) is real and positive for all x. Its Fourier cosine transform j,(k) 
is defined by 


ak) = [v(x eostkx) ds, 
and it is given that ~.(0) = 1. Prove that 
Fe(2k) > 2[Fe(k)P? — 1. 
17.2 Write the homogeneous Sturm-Liouville eigenvalue equation for which y(a) = 
y(b) =0 as 
L(y34) = (py') + ay + Apy = 0, 


where p(x), q(x) and p(x) are continuously differentiable functions. Show that if 
z(x) and F(x) satisfy £(z;A) = F(x), with z(a) = z(b) = 0, then 


ik y(x)F(x) dx = 0. 


Demonstrate the validity of this general result by direct calculation for the 
specific case in which p(x) = p(x) = 1, q(x) = 0, a 1,b =1 and z(x) =1—x°’. 
17.3 Consider the real eigenfunctions y,(x) of a Sturm—Liouville equation, 
(py') tay t+4py=0, axx<b, 


in which p(x), q(x) and p(x) are continuously differentiable real functions and 
p(x) does not change sign in a < x < b. Take p(x) as positive throughout the 
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17.4 


17.6 


17.8 


17.9 


interval, if necessary by changing the signs of all eigenvalues. For a < x; < x2 < b, 
establish the identity 
C—an) | PYnYm dx = [Yn PV, — Ym P Yn]. 

xX] 
Deduce that if 2, > 2, then y,(x) must change sign between two successive zeros 
of V(X). 
[The reader may find it helpful to illustrate this result by sketching the first few 
eigenfunctions of the system y” + Ay = 0, with y(0) = y(z) = 0, and the Legendre 
polynomials P,,(z) for n = 2,3,4,5.] 
Show that the equation 


y" +.a0(x)y + Ay = 0, 


with y(+z) = 0 and a real, has a set of eigenvalues 2 satisfying 
17 
tan(/2) a 2a 
a 
Investigate the conditions under which negative eigenvalues, 4 = —y’, with yw 
real, are possible. 
Use the properties of Legendre polynomials to carry out the following exercises. 


(a) Find the solution of (1 — x*)y” — 2xy’ + by = f(x), valid in the range 
—1<x<1 and finite at x = 0, in terms of Legendre polynomials. 

(b) If b = 14 and f(x) = 5x3, find the explicit solution and verify it by direct 
substitution. 


[The first six Legendre polynomials are listed in Subsection 18.1.1.] 

Starting from the linearly independent functions 1, x, x*, x*, ..., in the range 
0 <x < , find the first three orthogonal functions ¢0, ¢; and ¢2, with respect 
to the weight function p(x) = e-*. By comparing your answers with the Laguerre 
polynomials generated by the recurrence relation (18.115), deduce the form of 
$3(x). 

Consider the set of functions, {f(x)}, of the real variable x, defined in the interval 
—«0 <x <, that > 0 at least as quickly as x7! as x > +too. For unit weight 
function, determine whether each of the following linear operators is Hermitian 
when acting upon {f(x)}: 


d : _d 2. dd. @ 
(a) B+ 0) Sere (©) ixee (d) igs: 


A particle moves in a parabolic potential in which its natural angular frequency 
of oscillation is 3 At time t = 0 it passes through the origin with velocity v. It 
is then suddenly subjected to an additional acceleration, of +1 for 0 <t < 2/2, 
followed by —1 for 1/2 < t < a. At the end of this period it is again at the 


origin. Apply the results of the worked example in section 17.5 to show that 


8 1 
nr ome yD 
ca (4m +22 — 


T. 
m=0 4 
Find an eigenfunction expansion for the solution, with boundary conditions 
y(0) = y(z) = 0, of the inhomogeneous equation 
dy 


Ba thy = FO: 
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17.10 


17.11 


17.12 


17.13 


where x is a constant and 


j= {* 0<x<7/2, 


n—-x nt/2<x<n7. 


Consider the following two approaches to constructing a Green’s function. 


(a) Find those eigenfunctions y,(x) of the self-adjoint linear differential operator 
d°/dx* that satisfy the boundary conditions y,(0) = y,(z) = 0, and hence 
construct its Green’s function G(x, z). 

(b) Construct the same Green’s function using a method based on the comple- 
mentary function of the appropriate differential equation and the boundary 
conditions to be satisfied at the position of the 6-function, showing that it is 


_f xz—n)/n O<SxK<z, 
G2) = { zZ(x—n)/m z<x<70. 

(c) By expanding the function given in (b) in terms of the eigenfunctions y,(x), 
verify that it is the same function as that derived in (a). 


The differential operator £ is defined by 


= d dy 1x 
Ly =—— (« o) de. 


Determine the eigenvalues 2, of the problem 
LYn = An€Vn 0<x< 1, 
with boundary conditions 


dy 1 
y(0) =0, mt2=0 at x=1. 
(a) Find the corresponding unnormalised y,, and also a weight function p(x) with 
respect to which the y, are orthogonal. Hence, select a suitable normalisation 
for the y,. 


(b) By making an eigenfunction expansion, solve the equation 
Ly =e, 0<x<l, 
subject to the same boundary conditions as previously. 
Show that the linear operator 


44+ poate +4x(1+ oe +a. 
: dx? ? dx” 


acting upon functions defined in —1 < x < 1 and vanishing at the end-points of 
the interval, is Hermitian with respect to the weight function (1 +x?)1. 

By making the change of variable x = tan(@/2), find two even eigenfunctions, 
fi(x) and f(x), of the differential equation 


Lu = du. 


L 


By substituting x = expt, find the normalised eigenfunctions y,(x) and the 
eigenvalues 4, of the operator £ defined by 


Ly =xy" + 2xy' + 4y, 1<x<e, 


with y(1) = y(e) = 0. Find, as a series 7 dnyn(x), the solution of Ly = x71”. 
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17.14 


17.15 


17.3 


17.13 


17.15 


Express the solution of Poisson’s equation in electrostatics, 
V’ b(t) = —p(t)/€o, 
where p is the non-zero charge density over a finite part of space, in the form of 
an integral and hence identify the Green’s function for the V? operator. 
In the quantum-mechanical study of the scattering of a particle by a potential, 


a Born-approximation solution can be obtained in terms of a function y(r) that 
satisfies an equation of the form 


(—V? — K?)y(r) = F(r). 
Assuming that y,(r) = (27)~*/? exp(ik-r) is a suitably normalised eigenfunction of 
—V° corresponding to eigenvalue k’, find a suitable Green’s function Gx(r,1’). By 


taking the direction of the vector r—r’ as the polar axis for a k-space integration, 
show that Gx(r,r’) can be reduced to 


w sinw 
aim «o W2— we 
where wo = K|r—r'. 


[This integral can be evaluated using a contour integration (chapter 24) to give 
(4x|r — r'|)~! exp(iK |r — r'|).] 


17.8 Hints and answers 


Express the condition (h|h) > 0 as a quadratic equation in A and then apply the 
condition for no real roots, noting that (f|g) + (g|f) is real. To put a limit on 
f ycos? kx dx, set f = y' coskx and g = y!” in the inequality. 

Follow an argument similar to that used for proving the reality of the eigenvalues, 
but integrate from x; to x2, rather than from a to b. Take x; and x2 as two 
successive zeros of y,,(x) and note that, if the sign of y,, is « then the sign of y’,(x1) 
is « whilst that of y/,(x2) is —x. Now assume that y,(x) does not change sign in 
the interval and has a constant sign £; show that this leads to a contradiction 
between the signs of the two sides of the identity. 

= )¢ a,P,(x) with 


= —nt+l/2_ 
i ry ET b—n(n+1) [se P,,( 


(b) 5x? = 2P3(x)+3P,(x), giving a; = 1/4 and a3 = 1, leading to y = 5(2x°—x)/4. 
a) No, f gf*’dx # 0; (b) yes; (c) no, i f f* gdx #0; (d) yes. 
The normalised eigenfunctions are (2/z)'/? sin nx, with n an integer. 

y(x) = (4/2) >, cag l(—1)""? sin nx] /[n?(e — n7)]. 
Jn = (n+ 1/2)"x", n=0,1,2,... 
(a) Since y,(1)y/,(1) 4 0, the Sturm— Liouville boundary conditions are not satis- 
fied and the appropriate weight function has to be justified by inspection. The 
normalised eigenfunctions are J2e/? sin[(n + 1/2)zx], with p(x) = e*. 
(b) y(x) = (—2/0?) hg CO? sin[(n + 1/2)ax]/(n + 1/2). 
a(x) = /2x7!/? sin(nz In x) with 2, = —n?n?; 


ai2 (nn)? f* /2x sin(nz In x) dx = —,/8(nn)-> for n odd, 
- 0 for n even. 


Use the form of Green’s function that is the integral over all eigenvalues of the 
‘outer product’ of two eigenfunctions corresponding to the same eigenvalue, but 
with arguments r and r’. 
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Special functions 


In the previous two chapters, we introduced the most important second-order 
linear ODEs in physics and engineering, listing their regular and irregular sin- 
gular points in table 16.1 and their Sturm—Liouville forms in table 17.1. These 
equations occur with such frequency that solutions to them, which obey particu- 
lar commonly occurring boundary conditions, have been extensively studied and 
given special names. In this chapter, we discuss these so-called ‘special functions’ 
and their properties. In addition, we also discuss some special functions that are 
not derived from solutions of important second-order ODEs, namely the gamma 
function and related functions. These convenient functions appear in a number 
of contexts, and so in section 18.12 we gather together some of their properties, 
with a minimum of formal proofs. 


18.1 Legendre functions 
Legendre’s differential equation has the form 
(1 —x)y” — 2xy' + 7( + Dy =0, (18.1) 


and has three regular singular points, at x = —1,1,00. It occurs in numerous 
physical applications and particularly in problems with axial symmetry that 
involve the V? operator, when they are expressed in spherical polar coordinates. 
In normal usage the variable x in Legendre’s equation is the cosine of the polar 
angle in spherical polars, and thus —1 < x < 1. The parameter 7 is a given real 
number, and any solution of (18.1) is called a Legendre function. 

In subsection 16.1.1, we showed that x = 0 is an ordinary point of (18.1), and so 
we expect to find two linearly independent solutions of the form y = S77. 9 anx". 
Substituting, we find 

(ee) 
S [n(n — 1)ayx"? = n(n — L)ayx" — 2nayx" + C(¢ 4 Danx"] = 0, 

0 


n> 
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which on collecting terms gives 


ea 


So {2+ 2) + Vang — [nr + 1) — EE + Lan} x” = 0. 
n=0 


The recurrence relation is therefore 
[n(n + 1) —¢(¢ + 1)] 


= 18.2 
An+2 (n ee Dn +9) an, ( ) 
forn=0,1,2,.... If we choose aj = 1 and a; = 0 then we obtain the solution 
x? x4 
yi(x) =1-¢¢+ Na + (¢—2)¢(¢ + 1)(¢+ Ia eee, (18.3) 
whereas on choosing dy = 0 and a; = 1 we find a second solution 
x3 ae 
yo(x) =x-—(—-1)¢ + 237 + (¢ —3)(¢ —1)(¢ +2)(¢+ Yr —---, (18.4) 


By applying the ratio test to these series (see subsection 4.3.2), we find that both 
series converge for |x| < 1, and so their radius of convergence is unity, which 
(as expected) is the distance to the nearest singular point of the equation. Since 
(18.3) contains only even powers of x and (18.4) contains only odd powers, these 
two solutions cannot be proportional to one another, and are therefore linearly 
independent. Hence, the general solution to (18.1) for |x| < 1 is 


y(x) = cryi(x) + c2ya(x). 


18.1.1 Legendre functions for integer ¢ 


In many physical applications the parameter 7 in Legendre’s equation (18.1) is 
an integer, 1e. 7 = 0,1,2,.... In this case, the recurrence relation (18.2) gives 


e+ —¢¢ +1) 


4742 = +10 +2) l=Y%,; 

ie. the series terminates and we obtain a polynomial solution of order /. In 
particular, if 7 is even, then y;(x) in (18.3) reduces to a polynomial, whereas if / is 
odd the same is true of y2(x) in (18.4). These solutions (suitably normalised) are 
called the Legendre polynomials of order 7; they are written P;(x) and are valid 
for all finite x. It is conventional to normalise P/(x) in such a way that P/(1) = 1, 
and as a consequence P;(—1) = (—1)’. The first few Legendre polynomials are 
easily constructed and are given by 


Po(x) = 1, Pi(x) =x, 
Pp(x) = $(3x? — 1), P3(x) = 3(5x? — 3x), 
P4(x) = $(35x4 — 30x? + 3), Ps(x) = 3(63x° — 70x? + 15x). 
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Figure 18.1 The first four Legendre polynomials. 


The first four Legendre polynomials are plotted in figure 18.1. 

Although, according to whether / is an even or odd integer, respectively, either 
y1(x) in (18.3) or yo(x) in (18.4) terminates to give a multiple of the corresponding 
Legendre polynomial P,(x), the other series in each case does not terminate and 
therefore converges only for |x| < 1. According to whether / is even or odd, we 
define Legendre functions of the second kind as Q;(x) = %y2(x) or Q;(x) = By1(x), 
respectively, where the constants «, and f, are conventionally taken to have the 
values 


(—1)/?2’[(¢/2) 1? 
aa 7 
(DOV 2 Ne = 1)/2] 7 
7! 


for 7 even, (18.5) 


By for ¢ odd. (18.6) 


These normalisation factors are chosen so that the Q;(x) obey the same recurrence 
relations as the P,(x) (see subsection 18.1.2). 
The general solution of Legendre’s equation for integer ¢ is therefore 


V(X) = cr P/(x) + €20/(x), (18.7) 
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where P/(x) is a polynomial of order 7, and so converges for all x, and Q,(x) is 
an infinite series that converges only for |x| < 15 

By using the Wronskian method, section 16.4, we may obtain closed forms for 
the Q/(x). 


> Use the Wronskian method to find a closed-form expression for Qo(x). 


From (16.25) a second solution to Legendre’s equation (18.1), with 7 = 0, is 


x 1 u 2v 
rate) = Pos) fpr nw (fae) 


= if exp [—In(1 —u’)] du 


~ {43 
a yin(7**). (18.8) 


where in the second line we have used the fact that Po(x) = 1. 
All that remains is to adjust the normalisation of this solution so that it agrees with 
(18.5). Expanding the logarithm in (18.8) as a Maclaurin series we obtain 


a 962 


7) al a ee 


Comparing this with the expression for Qo(x), using (18.4) with 7 = 0 and the normalisation 
(18.5), we find that y2(x) is already correctly normalised, and so 


n(x) = gin (7**). 


Of course, we might have recognised the series (18.4) for 7 = 0, but to do so for larger 7 
would prove progressively more difficult. < 


Using the above method for 7 = 1, we find 
Q01(x) = 5x In (; **) —1. 


1-x 


Closed forms for higher-order Q/(x) may now be found using the recurrence 
relation (18.27) derived in the next subsection. The first few Legendre functions 
of the second kind are plotted in figure 18.2. 


18.1.2 Properties of Legendre polynomials 


As stated earlier, when encountered in physical problems the variable x in 
Legendre’s equation is usually the cosine of the polar angle 0 in spherical polar 
coordinates, and we then require the solution y(x) to be regular at x = +1, which 
corresponds to 0 = 0 or 0 = a. For this to occur we require the equation to have 
a polynomial solution, and so 7 must be an integer. Furthermore, we also require 


S It is possible, in fact, to find a second solution in terms of an infinite series of negative powers of 
x that is finite for |x| > 1 (see exercise 16.16). 
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5 


Figure 18.2 The first three Legendre functions of the second kind. 


the coefficient cz of the function Q,(x) in (18.7) to be zero, since Q,(x) is singular 
at x = +1, with the result that the general solution is simply some multiple of the 
relevant Legendre polynomial P,(x). In this section we will study the properties 
of the Legendre polynomials P,(x) in some detail. 


Rodrigues’ formula 


As an aid to establishing further properties of the Legendre polynomials we now 
develop Rodrigues’ representation of these functions. Rodrigues’ formula for the 
P, (x) is 
PAx) = sy 8 = 1) (189) 
OOO HEN dx 


To prove that this is a representation we let u = (x?—1)’, so that u! = 2/x(x?—1)/"! 
and 


(x? — 1)u' — 2¢/xu = 0. 
If we differentiate this expression / + 1 times using Leibnitz’ theorem, we obtain 


[(x? — Du) + 2x(¢ + Iu + 6 + Du] — 2¢ [xu*? + (6 + Tu] = 0, 
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which reduces to 
(x? = 1)u?) + 2xu!t) — e(¢ + tu = 0. 


Changing the sign all through, we recover Legendre’s equation (18.1) with u as 
the dependent variable. Since, from (18.9), 7 is an integer and u is regular at 
x = +1, we may make the identification 


u(x) = c,P/(x), (18.10) 


for some constant c, that depends on /. To establish the value of cy we note 
that the only term in the expression for the /th derivative of (x? — 1)’ that 
does not contain a factor x? — 1, and therefore does not vanish at x = 1, is 
(2x)/¢'(x? — 1)°. Putting x = 1 in (18.10) and recalling that P/(1) = 1, therefore 
shows that cy = 2’/!, thus completing the proof of Rodrigues’ formula (18.9). 


> Use Rodrigues’ formula to show that 


1 
ips J Pacorasyas rae (18.11) 


The result is trivially obvious for 7 = 0 and so we assume ¢ > 1. Then, by Rodrigues’ 


formula, 
1 vb d(x? — 1)/] [dé(x? — 1) 
I; == a Sy a ———_| dx. 
MeN? J, dx? dx! 


Repeated integration by parts, with all boundary terms vanishing, reduces this to 


1 
p= (—1/ i, (x? — 1) a (x? = 1)/ dx 


QC Jy dx? 
 QAb- fp 2 
= SETA [oes dx. 


If we write 
1 
K, -| (1 — x?) dx, 
-1 
then integration by parts (taking a factor 1 as the second part) gives 
1 
K; = ‘i 2¢x7(1 — x7)! dx. 
-1 


Writing 2/x? as 2¢ — 27(1 — x?) we obtain 


1 1 
K; x f (1—x°)1 dx x | (1 — x?) dx 
= -1 


1 


= 2/K,;_; — 2/K,; 
and hence the recurrence relation (27 + 1)K; = 2/K,;_,. We therefore find 
y he vyp\ WEI py)2 
K = 20 2 eK ave! 2C! >22 ans 
2+12/-1 3 (2¢ +1)! (2¢ + 1)! 


which, when substituted into the expression for I, establishes the required result. < 
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Mutual orthogonality 


In section 17.4, we noted that Legendre’s equation was of Sturm—Liouville form 
with p = 1— x*, q = 0, 4 = @(¢ +1) and p = 1, and that its natural interval 
was [—1, 1]. Since the Legendre polynomials P,(x) are regular at the end-points 
x = +1, they must be mutually orthogonal over this interval, i.e. 


1 
/ PAx)Pi(x)dx =0 if 24k. (18.12) 
= 


Although this result follows from the general considerations of the previous 
chapter, it may also be proved directly, as shown in the following example. 


>» Prove directly that the Legendre polynomials P;(x) are mutually orthogonal over the 
interval -1<x <1. 


Since the P/(x) satisfy Legendre’s equation we may write 
[(1—x?)P/]' + 2(¢ + 1)P, = 0, 


where P; = dP,/dx. Multiplying through by P, and integrating from x = —1 to x = 1, we 
obtain 
1 1 
/ Py [(1—3°)Ps]' dx + i P,¢(¢ + 1)P, dx = 0. 
-1 = 
Integrating the first term by parts and noting that the boundary contribution vanishes at 
both limits because of the factor 1 — x”, we find 


1 1 
-f P= x)Pedx-+ f P,¢(¢ + 1)P, dx = 0. 
I -l 


Now, if we reverse the roles of “ and k and subtract one expression from the other, we 
conclude that 


1 
[k(k +1) —¢(¢ + nf P,P, dx =0, 
-1 


and therefore, since k # 7, we must have the result (18.12). As a particular case, we note 
that if we put k = 0 we obtain 


1 
/ P/(x)dx =0 for? #0. 4 
=I 
As we discussed in the previous chapter, the mutual orthogonality (and com- 
pleteness) of the P/(x) means that any reasonable function f(x) (ie. one obeying 


the Dirichlet conditions discussed at the start of chapter 12) can be expressed in 
the interval |x| < 1 as an infinite sum of Legendre polynomials, 


f(x) = So ae P/(x), (18.13) 
where the coefficients a, are given by 


aa f(x)PAx) d (18.14) 
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> Prove the expression (18.14) for the coefficients in the Legendre polynomial expansion 


of a function f(x). 


If we multiply (18.13) by P;,(x) and integrate from x = —1 to x = 1 then we obtain 
1 co 1 
7 Profs) dx = So ay / P(x)P/(x) dx 
=f 0 vol 


1 
Si / _PalsPa(x) ds = 


2a, 
2k+1° 


where we have used the orthogonality property (18.12) and the normalisation property 
(18.11). < 


Generating function 


A useful device for manipulating and studying sequences of functions or quantities 
labelled by an integer variable (here, the Legendre polynomials P/(x) labelled by 
¢) is a generating function. The generating function has perhaps its greatest utility 
in the area of probability theory (see chapter 30). However, it is also a great 
convenience in our present study. 

The generating function for, say, a series of functions f,(x) for n =0,1,2,... is 
a function G(x,h) containing, as well as x, a dummy variable h such that 


G(x, h) = So faloph” 


n=0 
ie. f,(x) is the coefficient of h” in the expansion of G in powers of h. The utility 
of the device lies in the fact that sometimes it is possible to find a closed form 
for G(x, h). 
For our study of Legendre polynomials let us consider the functions P,,(x) 
defined by the equation 


G(x,h) =(1—2xh +h)? = 25 By (x)h". (18.15) 
n=0 
As we show below, the functions so defined are identical to the Legendre poly- 
nomials and the function (1 — 2xh +h*)~'/? is in fact the generating function for 
them. In the process we will also deduce several useful relationships between the 
various polynomials and their derivatives. 


> Show that the functions P,(x) defined by (18.15) satisfy Legendre’s equation 


In the following dP,,(x)/dx will be denoted by Pi. Firstly, we differentiate the defining 
equation (18.15) with respect to x and get 


h(l — 2xh +h?) 3? = =SoP, h". (18.16) 
Also, we differentiate (18.15) with respect to h to yield 
(x — h)(1 — 2xh + h?)3? = =~ nP,h"'. (18.17) 
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Equation (18.16) can then be written, using (18.15), as 
n> P,h” = (1—2xh+W?)>~ Pin, 
and equating the coefficients of h"*! we obtain the recurrence relation 
Py, = Phy; — 2xP,, + Pry. (18.18) 
Equations (18.16) and (18.17) can be combined as 


(x—h) $0 Pi = hd” Ph, 


from which the coefficent of h" yields a second recurrence relation, 


xP) — P/_, =nP,; (18.19) 
eliminating P’_, between (18.18) and (18.19) then gives the further result 
(n+ 1)P, = Pi, — xP. (18.20) 


If we now take the result (18.20) with n replaced by n—1 and add x times (18.19) to it 
we obtain 


(1 — x?)P! = n(P,_1 — xP). (18.21) 
Finally, differentiating both sides with respect to x and using (18.19) again, we find 
(1 —x°)P;’ — 2xP,, = n[(Py_1 — xP) — Pal 


= n(—nP,, ~ P,) _ —n(n + 1)P,,, 
and so the P,, defined by (18.15) do indeed satisfy Legendre’s equation. < 


The above example shows that the functions P,(x) defined by (18.15) satisfy 
Legendre’s equation with 7 = n (an integer) and, also from (18.15), these functions 
are regular at x = +1. Thus P, must be some multiple of the nth Legendre 
polynomial. It therefore remains only to verify the normalisation. This is easily 
done at x = 1, when G becomes 


Gh) = [L-hyy'? =1+htie+ 


and we can see that all the P,, so defined have P,,(1) = 1 as required, and are thus 
identical to the Legendre polynomials. 

A particular use of the generating function (18.15) is in representing the inverse 
distance between two points in three-dimensional space in terms of Legendre 
polynomials. If two points r and r’ are at distances r and 1’, respectively, from 
the origin, with r’ <r, then 

ise, 1 
lr—r'|  (r2 + 4? — 2rr’ cos 6)!/2 
_ 1 
r[1— 20) cos 0 + (r'/r)*]!/2 


a Due ) P,(cos 8), (18.22) 


where @ is the angle between the two position vectors r and r’. If r’ > r, however, 
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r and r’ must be exchanged in (18.22) or the series would not converge. This 
result may be used, for example, to write down the electrostatic potential at a 
point r due to a charge q at the point r’. Thus, in the case r’ <r, this is given by 


0 le 
q r 
V(r) = ree Dy (5) P,(cos 6). 


/=0 


We note that in the special case where the charge is at the origin, and r’ = 0, 
only the “ = 0 term in the series is non-zero and the expression reduces correctly 
to the familiar form V(r) = q/(4ze0r). 


Recurrence relations 


In our discussion of the generating function above, we derived several useful 
recurrence relations satisfied by the Legendre polynomials P,,(x). In particular, 
from (18.18), we have the four-term recurrence relation 


Phat +P =P, +2xP/. 


Also, from (18.19)-(18.21), we have the three-term recurrence relations 


Pi =(nt+ LP, +xP,, (18.23) 
P!_, =—nP, +xPi, (18.24) 
(1 — x?)P’ = n(P,_1 — XP,), (18.25) 
(2n + 1)Py = Phy) — Phy, (18.26) 


where the final relation is obtained immediately by subtracting the second from 
the first. Many other useful recurrence relations can be derived from those given 
above and from the generating function. 


> Prove the recurrence relation 


(n+ L)Prt = (2n+ 1)xP, — nPy-1. (18.27) 


Substituting from (18.15) into (18.17), we find 
(x—h) 5° Pyh" = (1—2xh +2?) S 7 nP,h". 
Equating coefficients of h” we obtain 
XP, — Pha = (n+ 1)Paai — 2xnP, + (n— 1)Ph-1, 


which on rearrangement gives the stated result. < 


The recurrence relation derived in the above example is particularly useful in 
evaluating P,,(x) for a given value of x. One starts with Po(x) = 1 and P,(x) =x 
and iterates the recurrence relation until P,,(x) is obtained. 
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18.2 Associated Legendre functions 


The associated Legendre equation has the form 
2 


(ashy 2 4 AC ED) y=0, (18.28) 


m 
1_ 
which has three regular singular points at x = —1, 1,00 and reduces to Legendre’s 
equation (18.1) when m = 0. It occurs in physical applications involving the 
operator V?, when expressed in spherical polars. In such cases, -/ < m < / and 
m is restricted to integer values, which we will assume from here on. As was the 
case for Legendre’s equation, in normal usage the variable x is the cosine of the 
polar angle in spherical polars, and thus —1 < x < 1. Any solution of (18.28) is 
called an associated Legendre function. 

The point x = 0 is an ordinary point of (18.28), and one could obtain series 
solutions of the form y = 0,9 4x" in the same manner as that used for 
Legendre’s equation. In this case, however, it is more instructive to note that if 
u(x) is a solution of Legendre’s equation (18.1), then 


diy 
dxim 


y(x) = (1 — 7)" —_ 


is a solution of the associated equation (18.28). 


(18.29) 


> Prove that if u(x) is a solution of Legendre’s equation, then y(x) given in (18.29) is a 
solution of the associated equation. 


For simplicity, let us begin by assuming that m is non-negative. Legendre’s equation for u 
reads 


(1 —x?)u"” — 2xu' + ¢2(2 + 1)u = 0, 
and, on differentiating this equation m times using Leibnitz’ theorem, we obtain 
(1 — x?)v" — 2x(m + 1)v' + (4 — m)\(¢ +m + Lv =0, (18.30) 
where v(x) = d"u/dx". On setting 
y(x) = (1 — x7)"7v(x), 
the derivatives v’ and v” may be written as 


v= ( = oe yale ( ’ Y) ; 
7 * mx m m(m + 2)x? 
WO ex 2)—m/2 
v” =(1—x°*) [»" boa ay tp mt (1x2? 
Substituting these expressions into (18.30) and simplifying, we obtain 


al 
Se = 


which shows that y is a solution of the associated ei equation (18.28). Finally, we 
note that if m is negative, the value of m? is unchanged, and so a solution for positive m 
is also a solution for the corresponding negative value of m. < 


(1 —x*)y” — 2xy’ + [a 


From the two linearly independent series solutions to Legendre’s equation given 
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in (18.3) and (18.4), which we now denote by u(x) and u2(x), we may obtain two 
linearly-independent series solutions, yi(x) and y2(x), to the associated equation 
by using (18.29). From the general discussion of the convergence of power series 
given in section 4.5.1, we see that both y,(x) and y2(x) will also converge for 
|x| < 1. Hence the general solution to (18.28) in this range is given by 


y(X) = cryi(x) + e2y2(x). 


18.2.1 Associated Legendre functions for integer ¢ 


If ¢ and m are both integers, as is the case in many physical applications, then 
the general solution to (18.28) is denoted by 


y(x) = e1P7"(x) + 6207), (18.31) 


where P/"(x) and Q?'(x) are associated Legendre functions of the first and second 
kind, respectively. For non-negative values of m, these functions are related to the 
ordinary Legendre functions for integer 7 by 

d"P, 


Pre one) =r 
x’ 


pr =({— 2)m/2 . 
t (x) ( x ) dx™ 


(18.32) 


We see immediately that, as required, the associated Legendre functions reduce 
to the ordinary Legendre functions when m = 0. Since it is m? that appears in 
the associated Legendre equation (18.28), the associated Legendre functions for 
negative m values must be proportional to the corresponding function for non- 
negative m. The constant of proportionality is a matter of convention. For the 
P/"(x) it is usual to regard the definition (18.32) as being valid also for negative m 
values. Although differentiating a negative number of times is not defined, when 
P,(x) is expressed in terms of the Rodrigues’ formula (18.9), this problem does 
not occur for —/ < m < 7S In this case, 
(¢—m)! 


P;"(x) = ey Game (18.33) 


> Prove the result (18.33). 


From (18.32) and the Rodrigues’ formula (18.9) for the Legendre polynomials, we have 


qétm 


(1 x?yn/2 


m 1 2 ¢ 
PMS) = 55 al =, 


and, without loss of generality, we may assume that m is non-negative. It is convenient to 


§ Some authors define P7"(x) = P7"(x), and similarly for the Q?'(x), in which case m is replaced by 
|m| in the definitions (18.32). It should be noted that, in this case, many of the results presented in 
this section also require m to be replaced by |mj. 
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write (x? — 1) = (x + 1)(x— 1) and use Leibnitz’ theorem to evaluate the derivative, which 
yields 


é+m 


2ym2 a ee (+m)! d'\(x+1) d""(x— 1) 


‘pm —_ OO 
Oy riZ+m—r)! = dx" dxé+m—r 


sn t! i 
Considering the two derivative factors in a term in the summation, we note that the first 


is non-zero only for r < / and the second is non-zero for 7+ m—r < ¢. Combining these 
conditions yields m <r < /. Performing the derivatives, we thus obtain 


m at EG 2yn/2 (¢ +m)! CWx+ i a ¢\(x — 1)" 
PA(x) = wal Lara (7—r)! Cm) 
mre +m)! (x + 1) THE (x — 1)? 
(—1) 2 ys ri¢tm—niZ—nir—m! (18.34) 


r=m 


Repeating the above calculation for P7"(x) and identifying once more those terms in 
the sum that are non-zero, we find 


¢—m m 


nate — oS _ Cet Fx yt 
rl 


Pe™(x) = (I) a (7 —m—ni?—nir+m)! 


onion ys (x+ 1H (x= IF 
2¢ (F—m)\(¢ —F)\(¢ +m—F) FV 


” 


(—1) 


(18.35) 


where, in the second equality, we have rewritten the summation in terms of the new index 
F =r+m. Comparing (18.34) and (18.35), we immediately arrive at the required result 
(18.33). < 


Since P(x) is a polynomial of order 7, we have P?"(x) = 0 for |m| > 7. From 
its definition, it is clear that P/"(x) is also a polynomial of order / if m is even, 
but contains the factor (1 — x?) to a fractional power if m is odd. In either case, 
P?'(x) is regular at x = +1. The first few associated Legendre functions of the 
first kind are easily constructed and are given by (omitting the m = 0 cases) 


P(x) =(1—x?)!?, P3(x) = 3x(1— x?)!/?, 
P3(x) = 3(1 — x?), P3(x) = 3 (5x? — 1)(1 — x7), 
P2(x) = 15x(1 — x?), P3(x) = 15(1 — x?)9/?, 


Finally, we note that the associated Legendre functions of the second kind Q?'(x), 
like Q,(x), are singular at x = +1. 


18.2.2 Properties of associated Legendre functions P?"(x) 


When encountered in physical problems, the variable x in the associated Legendre 
equation (as in the ordinary Legendre equation) is usually the cosine of the polar 
angle @ in spherical polar coordinates, and we then require the solution y(x) to 
be regular at x = +1 (corresponding to 0 = 0 or 0 = z). For this to occur, we 
require / to be an integer and the coefficient c) of the function Q?(x) in (18.31) 
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to be zero, since Q7'(x) is singular at x = +1, with the result that the general 
solution is simply some multiple of one of the associated Legendre functions of 
the first kind, P/"(x). We will study the further properties of these functions in 
the remainder of this subsection. 


Mutual orthogonality 


As noted in section 17.4, the associated Legendre equation is of Sturm—Liouville 
form (py) + qy + Apy = 0, with p = 1—x?, q = —m’/(1— x2), 2 = &¢ +1) 
and p = 1, and its natural interval is thus [—1, 1]. Since the associated Legendre 
functions P/"(x) are regular at the end-points x = +1, they must be mutually 
orthogonal over this interval for a fixed value of m, i.e. 


1 
io PMx)PMx)dx=0 if 0k. (18.36) 


This result may also be proved directly in a manner similar to that used for demon- 
strating the orthogonality of the Legendre polynomials P,(x) in section 18.1.2. 
Note that the value of m must be the same for the two associated Legendre 
functions for (18.36) to hold. The normalisation condition when ¢ = k may be 
obtained using the Rodrigues’ formula, as shown in the following example. 


> Show that 


2 (¢+m)! 


~ 441 ¢—m! eae 


From the definition (18.32) and the Rodrigues’ formula (18.9) for P;(x), we may write 


1 1 we d’ +(x? we 1) alt (x? cae 1) 
Lin = (C12 I la —x’) dx/+m dx+m dx, 


where the square brackets identify the factors to be used when integrating by parts. 
Performing the integration by parts “ +m times, and noting that all boundary terms 
vanish, we obtain 


1)" aa d+” (x? = 1) 
2h m 
Lom = aaa = +f i Y cam a — x?) dxeem dx. 


Using Leibnitz’ theorem, the second factor in the integrand may be written as 


diem ey d’t" (x? - a 3 (¢ + m)! a(t Sa gee t2m—r (x2 es 1)’ 
dx/+m dxé+m r\(¢+m—r)! dx" dx2¢+2m—r 
Considering the two derivative factors in a term in the summation on the RHS, we 
see that the first is non-zero only for r < 2m, whereas the second is non-zero only for 
2¢+2m—r < 2¢. Combining these conditions, we find that the only non-zero term in the 
sum is that for which r = 2m. Thus, we may write 


(-1)" (¢ + m)! i 1 A per SS. x?y" (1 ae, x?) 
)! 


DEN mye — mi ft * am ae 


Tin 
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Since d?/(1 — x*)’/dx*” = (—1)‘(2/)!, and noting that (—1)?/+?”" = 1, we have 


1 (20)"¢ +m)! 
Pye ¢—m! [urs 


Iq = 


We have already shown in section 18.1.2 that 


1 Pole!) 
= 2\¢ 
Ke fa x°) dx Fe)! 


and so we obtain the final result 


1. = 2 etm)! 
™™ W+1C—m! 


The orthogonality and normalisation conditions, (18.36) and (18.37) respec- 
tively, mean that the associated Legendre functions P/"(x), with m fixed, may be 
used in a similar way to the Legendre polynomials to expand any reasonable 
function f(x) on the interval |x| < 1 in a series of the form 


f(%) = So ams Pr (*), (18.38) 


k=0 


where, in this case, the coefficients are given by 


2¢+1(¢—m) 


ag = a Caimi va frei (x) dx. 


We note that the series takes the form (18.38) because P/"(x) = 0 for m> 7. 
Finally, it is worth noting that the associated Legendre functions P/”(x) must 
also obey a second orthogonality relationship. This has to be so because one may 
equally well write the associated Legendre equation (18.28) in Sturm—Liouville 
form (py)'+qy+4py = 0, with p = 1—x?, q = ¢(¢+1), 2 = —m? and p = (1—x’)"!; 
once again the natural interval is [—1, 1]. Since the associated Legendre functions 


P?"(x) are regular at the end-points x = +1, they must therefore be mutually 
orthogonal with respect to the weight function (1 — x*)~! over this interval for a 
fixed value of /, ie. 


1 
/ P(x)PK(x)\(1—x?)tdx =0 if |m| 4 |kl. (18.39) 
One may also show straightforwardly that the corresponding normalisation con- 
dition when m =k is given by 


_ (+m)! 


1 
m my. 2)-1 
pe CPP OOU = x ax = 


In solving physical problems, however, the orthogonality condition (18.39) is not 
of any practical use. 
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Generating function 


The generating function for associated Legendre functions can be easily derived 
by combining their definition (18.32) with the generating function for the Legendre 
polynomials given in (18.15). We find that 


(2m)!(1 — x2ym/2 oe 
me 


G(x,h) = 
(% 1) = Simi — Dlx + Dra 


P m 


n+m 


(x)h", (18.40) 
n=0 


> Derive the expression (18.40) for the associated Legendre generating function. 


The generating function (18.15) for the Legendre polynomials reads 


>> Ph" = (1— 2xh +, 


n=0 


Differentiating both sides of this result m times (assumimg m to be non-negative), mutli- 
plying through by (1 — x?)"”? and using the definition (18.32) of the associated Legendre 
functions, we obtain 


a) qn 
3. PMH = (1 x7"? (1 = xh + YY, 
a dx™ 


Performing the derivatives on the RHS gives 


s pap 1-3-5--+(2m—1)(1 — x2)"/?2n 
n (1 —2xh + h2ym+1/2 


n=0 


Dividing through by h”, re-indexing the summation on the LHS and noting that, quite 
generally, 


1-3-5--Qr—-l= 


we obtain the final result (18.40). < 


Recurrence relations 


As one might expect, the associated Legendre functions satisfy certain recurrence 
relations. Indeed, the presence of the two indices n and m means that a much wider 
range of recurrence relations may be derived. Here we shall content ourselves 
with quoting just four of the most useful relations: 


pm — ee + [m(m— 1) —n(n + 1)]P”1, (18.41) 

(2n + 1)xP” =(n+m)P, +(n—m+t 1PM, (18.42) 

(2n+ 1)1—x°)? Pm = prt! — pmtl, (18.43) 
21 — x?)/?2(Pp™y = P™! (n+ m)(n—mt1)P™, (18.44) 


We note that, by virtue of our adopted definition (18.32), these recurrence relations 
are equally valid for negative and non-negative values of m. These relations may 
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be derived in a number of ways, such as using the generating function (18.40) or 
by differentiation of the recurrence relations for the Legendre polynomials P;(x). 


> Use the recurrence relation (2n+1)P, =P), —P)_, for Legendre polynomials to derive 


the result (18.43 ). 


Differentiating the recurrence relation for the Legendre polynomials m times, we have 


d"P,, a"! Py a™*!p 
(2n + 1) dx = dxmtl - dxmtl * 


Multiplying through by (1 — x?)"*+/? and using the definition (18.32) immediately gives 
the result (18.43). < 


18.3. Spherical harmonics 


The associated Legendre functions discussed in the previous section occur most 
commonly when obtaining solutions in spherical polar coordinates of Laplace’s 
equation Vu = 0 (see section 21.3.1). In particular, one finds that, for solutions 
that are finite on the polar axis, the angular part of the solution is given by 


©(0)®(d) = P7"(cos 0)(C cosm@ + D sinm®@), 


where / and m are integers with —/ < m < /. This general form is sufficiently 
common that particular functions of 0 and ¢ called spherical harmonics are 
defined and tabulated. The spherical harmonics Y/"(0, %) are defined by 


2/+1(¢—m)! 
4n (¢+m)! 


1/2 
Y;"(0, b) = (—1)" | | P;"(cos 0) exp(im@). (18.45) 


Using (18.33), we note that 


¥7"(0, 9) = (—1)" [¥/"(8, 9) 


where the asterisk denotes complex conjugation. The first few spherical harmonics 
Y/"(0,~) = Y/" are as follows: 


* 
> 


0 1 0_ 3 
Yo iv Yp = F 0080, 


Yel =  Fy/ gq sinOexp(tid), Y= \/ 6:3. c0s’ 0 — 1), 
ea F4/ ze sin 0 cos 0 exp(+i¢), Ys = — sin’ 0 exp(+2i¢). 


Since they contain as their 0-dependent part the solution P/” to the associated 
Legendre equation, the Y/” are mutually orthogonal when integrated from —1 to 
+1 over d(cos@). Their mutual orthogonality with respect to ¢ (0 < ¢ < 2z) is 
even more obvious. The numerical factor in (18.45) is chosen to make the Y/" an 


ll 


ll 


593 


SPECIAL FUNCTIONS 


orthonormal set, Le. 


2n 
i. chat /"(0,)| Y2" (0, &) dp d(cos 0) = 5/7b inn: (18.46) 


In addition, the spherical harmonics form a complete set in that any reasonable 
function (i.e. one that is likely to be met in a physical situation) of @ and ¢ can 
be expanded as a sum of such functions, 


ive) ¢ 
£(0.6) = S- So amY/(0. 0), (18.47) 


¢=0 m=—¢ 


the constants a,,, being given by 


2n 
aim = a [ y;"( | £6, ob) dd d(cos 0). (18.48) 


This is in exact analogy with a Fourier series and is a particular example of the 
general property of Sturm—Liouville solutions. 

Aside from the orthonormality condition (18.46), the most important relation- 
ship obeyed by the Y/” is the spherical harmonic addition theorem. This reads 


P,(cosy) = 


s ¥"(0, o)(Y"(0.@1". (18.49) 


m=—¢ 


rai 


where (0, ¢) and (6’, o’) denote two different directions in our spherical polar coor- 
dinate system that are separated by an angle y. In general, spherical trigonometry 
(or vector methods) shows that these angles obey the identity 


cosy = cos@ cos 6’ + sin @ sin 6’ cos(¢ — ¢’). (18.50) 


& Prove the spherical harmonic addition theorem (18.49). 


For the sake of brevity, it will be useful to denote the directions (0,@) and (0’,¢’) by 
Q and ’, respectively. We will also denote the element of solid angle on the sphere by 
dQ = d¢ d(cos 0). We begin by deriving the form of the closure relationship obeyed by the 
spherical harmonics. Using (18.47) and (18.48), and reversing the order of the summation 
and integration, we may write 


fa) = ff ase fay very") 


ém 


where )_,,,, is a convenient shorthand for the double summation in (18.47). Thus we may 
write the closure relationship for the spherical harmonics as 


pees MQVY, 7m (O!) = 6(Q—'), (18.51) 


ém 


where 6(Q —{’) is a Dirac delta function with the properties that 6(Q —Q’) = 0 if Q 4’ 
and f,, 6(Q)dQ = 1. 
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Since 6(Q — ’) can depend only on the angle y between the two directions Q and 0’, 
we may also expand it in terms of a series of Legendre polynomials of the form 


6(Q—-O') = 5 b,P-(cos y). (18.52) 
7 
From (18.14), the coefficients in this expansion are given by 
bp = “et i 6(Q — Q')P;(cos y) d(cos y) 
2+ 2 


4n 


where, in the second equality, we have introduced an additional integration over an 
azimuthal angle y about the direction 9’ (and y is now the polar angle measured from 
Q’ to Q). Since the rest of the integrand does not depend upon y, this is equivalent 
to multiplying it by 27/22. However, the resulting double integral now has the form of 
a solid-angle integration over the whole sphere. Moreover, when Q = ’, the angle » 
separating the two directions is zero, and so cosy = 1. Thus, we find 

+1 2e+1 


be Fm Pell) ae 


and combining this expression with (18.51) and (18.52) gives 


S> ¥7Q)Y7"(Q) = >> at Le eben (18.53) 
fm t 


2n 
= = [ [ 6(Q—Q')P;(cos y) d(cos y) dy, 


Comparing this result with (18.49), we see that, to complete the proof of the addition 
theorem, we now only need to show that the summations in / on either side of (18.53) can 
be equated term by term. 

That such a procedure is valid may be shown by considering an arbitrary rigid rotation 
of the coordinate axes, thereby defining new spherical polar coordinates Q on the sphere. 
Any given spherical harmonic Y/"(Q) in the new coordinates can be written as a linear 
combination of the spherical ihacrionies Y/"(Q) of the old coordinates, all having the same 
value of 7. Thus, 


¥"(Q) = = pa" yy" (Q), 
nl! = 
where the coefficients Din depend on the rotation; note that in this expression Q and Q 
refer to the same direction, but expressed in the two different coordinate systems. If we 


choose the polar axis of the new coordinate system to lie along the 1?’ direction, then from 
(18.45), with m in that equation set equal to zero, we may write 


An = Foy 
Pr(cosy) = yf 57 VPQ) = D7 CM Y?(Q) 


m'=—¢ 


for some set of coefficients C2” that depend on ’. Thus, we see that the equality (18.53) 
does indeed hold term by term in /, thus proving the addition theorem (18.49). < 


18.4 Chebyshev functions 


Chebyshev’s equation has the form 


(1 —x*)y” — xy’ + v?y =0, (18.54) 


SPECIAL FUNCTIONS 


and has three regular singular points, at x = —1,1,00. By comparing it with 
(18.1), we see that the Chebyshev equation is very similar in form to Legendre’s 
equation. Despite this similarity, equation (18.54) does not occur very often 
in physical problems, though its solutions are of considerable importance in 
numerical analysis. The parameter v is a given real number, but in nearly all 
practical applications it takes an integer value. From here on we thus assume 
that v = n, where n is a non-negative integer. As was the case for Legendre’s 
equation, in normal usage the variable x is the cosine of an angle, and so 
—1<x <1. Any solution of (18.54) is called a Chebyshev function. 

The point x = 0 is an ordinary point of (18.54), and so we expect to find 
two linearly independent solutions of the form y = >) dmx”. One could find 
the recurrence relations for the coefficients a,, in a similar manner to that used 
for Legendre’s equation in section 18.1 (see exercise 16.15). For Chebyshev’s 
equation, however, it is easier and more illuminating to take a different approach. 
In particular, we note that, on making the substitution x = cos 0, and consequently 
d/dx = (—1/sin 0) d/d0, Chebyshev’s equation becomes (with v = n) 

Wy 


Be me 
qoz Thy =9 


which is the simple harmonic equation with solutions cosn@ and sinn@. The 
corresponding linearly independent solutions of Chebyshev’s equation are thus 
given by 

T,(x) = cos(ncos-!x) and V,(x) = sin(ncos~! x). (18.55) 
It is straightforward to show that the T,,(x) are polynomials of order n, whereas 
the V,,(x) are not polynomials 


> Find explicit forms for the series expansions of T,(x) and V,,(x). 


Writing x = cos 0, it is convenient first to form the complex superposition 
T,.(x) + iV, (x) = cos n@ + isin nd 
= (cos @ + isin 6)" 


n 
= (x +ivT—x) for |x| < 1. 
Then, on expanding out the last expression using the binomial theorem, we obtain 


T(x) =x" "Cyx" (1 =, x?) + "Cyx" 41 = x)? they (18.56) 


V,,(x) = Jfox ["Cyx" _ "C3x"™3(1 = x?) + "Csx"(1 = x?) ne | ; (18.57) 


where "C, = n!/[r!(n—r)!] is a binomial coefficient. We thus see that T,,(x) is a polynomial 
of order n, but V,,(x) is not a polynomial. < 


It is conventional to define the additional functions 
W(x) =(1— x)? Tyt(x) and p(x) = (1 — x?) Vat (2). 
(18.58) 
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Figure 18.3 The first four Chebyshev polynomials of the first kind. 


From (18.56) and (18.57), we see immediately that U,,(x) is a polynomial of order 
n, but that W,,(x) is not a polynomial. In practice, it is usual to work entirely in 
terms of T,,(x) and U,(x), which are known, respectively, as Chebyshev polynomials 
of the first and second kind. In particular, we note that the general solution to 
Chebyshev’s equation can be written in terms of these polynomials as 


¢1T,(x) + coJ1 — x2 U,_1(x) for n = 1,2,3,..., 


cr +osin ! x forn=0. 


y(x) = 


The n = 0 solution could also be written as dj +c. cos! x with dj = c; + ZC. 
The first few Chebyshev polynomials of the first kind are easily constructed 
and are given by 


To(x) = 1, T(x) =x, 
T(x) = 2x? — 1, T3(x) = 4x3 — 3x, 
Ta(x) = 8x* — 8x? +1, Ts(x) = 16x5 — 20x3 + 5x. 


The functions T(x), T(x), T(x) and T3(x) are plotted in figure 18.3. In general, 
the Chebyshev polynomials T,,(x) satisfy T,(—x) = (—1)"T,,(x), which is easily 
deduced from (18.56). Similarly, it is straightforward to deduce the following 
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Figure 18.4 The first four Chebyshev polynomials of the second kind. 


special values: 
T,(1) = 1, T,(—1) = (-1)", Tn(0) =(—1)", Tan 41(0) = 0. 


The first few Chebyshev polynomials of the second kind are also easily found 
and read 


Uo(x) = 1, U;(x) = 2x, 
U2(x) = 4x? — 1, U3(x) = 8x3 — 4x, 
U4(x) = 16x4 — 12x? + 1, Us(x) = 32x> — 32x3 + 6x. 


The functions Uo(x), Ui(x), U2(x) and U3(x) are plotted in figure 18.4. The 
Chebyshev polynomials U,,(x) also satisfy U,(—x) = (—1)"U,,(x), which may be 
deduced from (18.57) and (18.58), and have the special values: 


Un) =n+1, U,(—1) = (-1)"(n + 1), U2n(0) = (—1)", Urn+1(0) = 0. 


> Show that the Chebyshev polynomials U,,(x) satisfy the differential equation 
(1 — x°)U"(x) — 3xU} (x) + n(n + 2)U,(x) = 0. (18.59) 


From (18.58), we have Visi = (1 — x?)!/U, and these functions satisfy the Chebyshev 
equation (18.54) with v =n+ 1, namely 


(122), = XV, + (nt Png = 0. (18.60) 
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Evaluating the first and second derivatives of V,+1, we obtain 


Viv el x y?U x(1 xy? U, 


Vit = (L—32)?U8 — 2x(1 — 21?) — (1 — 2), — 21 — 2), 


1/2 


Substituting these expressions into (18.60) and dividing through by (1 — x?)!/?, we find 


(1 —x°)U” — 3xU/ — U, + (n+ 1)U, =0, 


which immediately simplifies to give the required result (18.59). < 


18.4.1 Properties of Chebyshev polynomials 


The Chebyshev polynomials T,(x) and U,(x) have their principal applications 
in numerical analysis. Their use in representing other functions over the range 
|x| < 1 plays an important role in numerical integration; Gauss—Chebyshev 
integration is of particular value for the accurate evaluation of integrals whose 
integrands contain factors (1 — x?)*!/?. It is therefore worthwhile outlining some 
of their main properties. 


Rodrigues’ formula 


The Chebyshev polynomials T,,(x) and U,(x) may be expressed in terms of a 
Rodrigues’ formula, in a similar way to that used for the Legendre polynomials 
discussed in section 18.1.2. For the Chebyshev polynomials, we have 


_ (—1)",/a(1 = x2)1/2 d@ 5 nt 
Ta(x) = nay ax —x) 2, 
U,(x) = (—1)" /x(n a 1) d" (1 = eyed, 


21 (yn 4 5)i(1 — x2)1/2 dx" 


These Rodrigues’ formulae may be proved in an analogous manner to that used 
in section 18.1.2 when establishing the corresponding expression for the Legendre 
polynomials. 


Mutual orthogonality 


In section 17.4, we noted that Chebyshev’s equation could be put into Sturm— 
Liouville form with p = (1 —x?)/?, q=0, 4 = 9? and p = (1—x?)-'”, and its 
natural interval is thus [—1, 1]. Since the Chebyshev polynomials of the first kind, 
T,(x), are solutions of the Chebyshev equation and are regular at the end-points 
x = +1, they must be mutually orthogonal over this interval with respect to the 
weight function p = (1 — x*)~!?, ie. 


1 
if Ti(X) T(x) — x?) 2 dx =0 ifn #m. (18.61) 
1 
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The normalisation, when m = n, is easily found by making the substitution 
x =cos0@ and using (18.55). We immediately obtain 


1 fe =0 
i: T(x) To(x)(1 — 22-2 dx = 97 OEE” 
-1 n/2 forn=1,2,3,.... (18.62) 


The orthogonality and normalisation conditions mean that any (reasonable) 
function f(x) can be expanded over the interval |x| < 1 in a series of the form 


fle) = $ay + Yo ay Tal), 


n=1 


where the coefficients in the expansion are given by 


1 
a == | fO)T,(x)(1 — 2)? dx, 
4 


For the Chebyshev polynomials of the second kind, U,,(x), we see from (18.58) 
that (1 — x?)!/?U,,(x) = Vy4i(x) satisfies Chebyshev’s equation (18.54) with v = 
n+ 1. Thus, the orthogonality relation for the U,(x), obtained by replacing T;(x) 
by Vj41(x) in equation (18.61), reads 


1 
ih U,(x)Um(x)(1 — x”)/? dx = 0 ifn #m. 
4 


The corresponding normalisation condition, when n = m, can again be found by 
making the substitution x = cos 0, as illustrated in the following example. 


> Show that 


From (18.58), we see that 
1 
po 7 Voss) Vasil — 2)? dx, 
-1 
which, on substituting x = cos 0, gives 


0 
d -| sin(n + 1)0 sin(n + 1)9 —_ (—sin0) a0 =..<4 
fo sin 0 


NIA 


The above orthogonality and normalisation conditions allow one to expand 
any (reasonable) function in the interval |x| < 1 in a series of the form 


f(x) = > a,U,,(x), 


n=0 
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in which the coefficients a, are given by 
1 
nee i. FOU, (x1 — 2)? dx. 
RX S-1 


Generating functions 


The generating functions for the Chebyshev polynomials of the first and second 
kinds are given, respectively, by 


1—xh ke 7 

Gi) = 75 p= 2 Ta(x)h", (18.63) 
1 n 

Gul) = 5 = S5 Un (x)h". (18.64) 


n=0 


These prescriptions may be proved in a manner similar to that used in sec- 
tion 18.1.2 for the generating function of the Legendre polynomials. For the 
Chebyshev polynomials, however, the generating functions are of less practical 
use, since most of the useful results can be obtained more easily by taking 
advantage of the trigonometric forms (18.55), as illustrated below. 


Recurrence relations 


There exist many useful recurrence relationships for the Chebyshev polynomials 
T,(x) and U,(x). They are most easily derived by setting x = cos@ and using 
(18.55) and (18.58) to write 


T,(x) = T,(cos 0) = cos nd, (18.65) 


sin(n + 1)0 (18.66) 


U,(x) = U,(cos 0) = ind 


One may then use standard formulae for the trigonometric functions to derive 
a wide variety of recurrence relations. Of particular use are the trigonometric 
identities 
cos(n + 1)0 = cosné cos @ + sinné sin 0, (18.67) 
sin(n + 1)0 = sinnd cos@ + cosné sin 0. (18.68) 


> Show that the Chebyshev polynomials satisfy the recurrence relations 


Thai (%) — 2xT,(x) + Tr-1(x) = 0, 


Un41(%) = 2xU n(x) + Una(x) = 0. 


Adding the result (18.67) with the plus sign to the corresponding result with a minus sign 
gives 


cos(n + 1)@ + cos(n — 1)@ = 2cosné@ cos 0. 
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Using (18.65) and setting x = cos@ immediately gives a rearrangement of the required 
result (18.69). Similarly, adding the plus and minus cases of result (18.68) gives 


sin(n + 1)@ + sin(n — 1)0 = 2sinn@ cos @. 
Dividing through on both sides by sin @ and using (18.66) yields (18.70). < 


The recurrence relations (18.69) and (18.70) are extremely useful in the practical 
computation of Chebyshev polynomials. For example, given the values of To(x) 
and T(x) at some point x, the result (18.69) may be used iteratively to obtain 
the value of any T,,(x) at that point; similarly, (18.70) may be used to calculate 
the value of any U,(x) at some point x, given the values of Uo(x) and Uj(x) at 
that point. 

Further recurrence relations satisfied by the Chebyshev polynomials are 


T(x) = Uy (x) — XUp-1(%), (18.71) 

(1 — x°)U n(x) = xT nga) — Taal), (18.72) 

which establish useful relationships between the two sets of polynomials T,,(x) 
and U,,(x). The relation (18.71) follows immediately from (18.68), whereas (18.72) 
follows from (18.67), with n replaced by n+ 1, on noting that sin? @ = 1 — x’. 


Additional useful results concerning the derivatives of Chebyshev polynomials 
may be obtained from (18.65) and (18.66), as illustrated in the following example. 


> Show that 
T, (x) = nU,1(x), 
(1 = x2)UL(x) = xUn(x) — (0 + 1) Tra. 


These results are most easily derived from the expressions (18.65) and (18.66) by noting 
that d/dx = (—1/sin 0) d/d0. Thus, 


1 d(cosn0) — nsinn@ 
sin@  d0 sin 0 


nU,_1(X). 


Similarly, we find 


U'(x) 1 od sin(n + 1)0 sin(n+1)0 cos@ (n+1)cos(n + 1)0 


sin 6 d@ sin 0 sin? 0 sin’ @ 
XU) (n+) Tri) 
~ [x 1—x?2 : 


which rearranges immediately to yield the stated result. < 


18.5 Bessel functions 
Bessel’s equation has the form 


, 
i 


x?y"” + xy" + (x? — vy = 0, (18.73) 


which has a regular singular point at x = 0 and an essential singularity at x = 00. 
The parameter v is a given number, which we may take as > 0 with no loss of 
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generality. The equation arises from physical situations similar to those involving 
Legendre’s equation but when cylindrical, rather than spherical, polar coordinates 
are employed. The variable x in Bessel’s equation is usually a multiple of a radial 
distance and therefore ranges from 0 to o. 

We shall seek solutions to Bessel’s equation in the form of infinite series. Writing 
(18.73) in the standard form used in chapter 16, we have 


ieee L , | MS — 
y t-yt+ {1 y=0. (18.74) 
x 


x2 
By inspection, x = 0 is a regular singular point; hence we try a solution of the 
form y = x7 yy a,x". Substituting this into (18.74) and multiplying the resulting 
equation by x?~°, we obtain 


a 


> [(o +n)(o +n—1) + (6 +1) —V?| ayx" + Sax"? =0, 
n=0 


n=0 


which simplifies to 


ee) 


> [(o + ny = v7] AyX" + yy aux? = 0. 


n=0 n=0 
Considering the coefficient of x°, we obtain the indicial equation 
o—v=0, 
and so o = +v. For coefficients of higher powers of x we find 
[(o + 1)? —v"] a1 =0, (18.75) 
[(o +n) —v*] da, +a,» =0 forn>2. (18.76) 
Substituting o = +v into (18.75) and (18.76), we obtain the recurrence relations 
(1+ 2v)a, =0, (18.77) 
n(n 2v)ady +Q,-2=0  forn> 2. (18.78) 


We consider now the form of the general solution to Bessel’s equation (18.73) for 
two cases: the case for which v is not an integer and that for which it is (including 
Zero). 


18.5.1 Bessel functions for non-integer v 


If v is a non-integer then, in general, the two roots of the indicial equation, 
o, =v and o> = —v, will not differ by an integer, and we may obtain two linearly 
independent solutions in the form of Frobenius series. Special considerations do 
arise, however, when v = m/2 for m = 1,3,5,..., and o; — 0. = 2v =m is an 
(odd positive) integer. When this happens, we may always obtain a solution in 
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the form of a Frobenius series corresponding to the larger root, 01 = v = m/2, 
as described above. However, for the smaller root, a7 = —v = —m/2, we must 
determine whether a second Frobenius series solution is possible by examining 
the recurrence relation (18.78), which reads 


n(n—m)a, + a,-2=0 for n> 2. 


Since m is an odd positive integer in this case, we can use this recurrence relation 
(starting with ao # 0) to calculate az, a4,d6,... in the knowledge that all these 
terms will remain finite. It is possible in this case, therefore, to find a second 
solution in the form of a Frobenius series, one that corresponds to the smaller 
root 02. 

Thus, in general, for non-integer v we have from (18.77) and (18.78) 


1 
a, = —-——~~42 forn=2,4,6,..., 
n n(n + 2v) n—2 
= 0 for n= 1,3,5,.... 
Setting aj = | in each case, we obtain the two solutions 
sty x oa 


Y(x) = x7" J 1 22L2v) 1 2x4 QE WL) 


It is customary, however, to set 


1 
2 T(.+v)’ 
where I(x) is the gamma function, described in subsection 18.12.1; it may be 
regarded as the generalisation of the factorial function to non-integer and/or 
negative arguments.$ The two solutions of (18.73) are then written as J,(x) and 
J_,(x), where 


o0= aren) |! @) + penta @) -~ 


(—1)" vn 
=a @) ? (18.79) 


replacing v by —v gives J_,(x). The functions J,(x) and J_,(x) are called Bessel 
functions of the first kind, of order v. Since the first term of each series is a 
finite non-zero multiple of x” and x~’, respectively, if v is not an integer then 
J\(x) and J_,(x) are linearly independent. This may be confirmed by calculating 
the Wronskian of these two functions. Therefore, for non-integer v the general 
solution of Bessel’s equation (18.73) is given by 


y(x) = c1Jy(x) + c2J_y(X). (18.80) 


a = 


S In particular, P(n+ 1) =n! for n= 0,1,2,..., and I'(n) is infinite if n is any integer < 0. 
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We note that Bessel functions of half-integer order are expressible in closed form 
in terms of trigonometric functions, as illustrated in the following example. 


> Find the general solution of 
x?y" + xy! +(x? — Z)y = 0. 


This is Bessel’s equation with v = 1/2, so from (18.80) the general solution is simply 
V(X) = e11/2(X) + c2J-1/2(X). 


However, Bessel functions of half-integral order can be expressed in terms of trigonometric 
functions. To show this, we note from (18.79) that 


ny.2n 
_ +1/2 (-1)"x 
Jxij2(x) = x > 22nt1/2n IT ( (nat) $1) 


Using the fact that I(x + 1) = x(x) and I'( ()= ./m, we find that, for v = 1/2, 


(yl? dx? (x? 


ra) org) ir) 
(3x x)!/? ($x)? (4x)? 
(4x)!/? xed (4x)!/? sin x 24 
(jm (1 ar) 5 ) G)je x Vax 
whereas for v = —1/2 we obtain 
1y)-1/2 1.3/2 1y.)7/2 
Fapte = G22 GPF, Go? 


T(3) 13) 2!) 


= (4x)? 1 x? ; x4 [Zcosx 
Ja 2!" 4! as 


Therefore the general solution we require is 


EQ / 2 
y(X) = cy J1/2(X) + c2J_1/2(x) = €14/ — sin x + c24/ — cosx. < 
TX 1X 


18.5.2 Bessel functions for integer v 


The definition of the Bessel function J,(x) given in (18.79) is, of course, valid for 
all values of v, but, as we shall see, in the case of integer v the general solution of 
Bessel’s equation cannot be written in the form (18.80). Firstly, let us consider the 
case v = 0, so that the two solutions to the indicial equation are equal, and we 
clearly obtain only one solution in the form of a Frobenius series. From (18.79), 
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Figure 18.5 The first three integer-order Bessel functions of the first kind. 


this is given by 
® (—1)"x2" 
J = ees 
o() X, nl +n) 
x? x4 x6 
72 7242 324262 


In general, however, if v is a positive integer then the solutions of the indicial 


=1 San 


equation differ by an integer. For the larger root, 0, = v, we may find a solution 
J\(x), for v = 1,2,3,..., in the form of the Frobenius series given by (18.79). 
Graphs of Jo(x), J:(x) and J>(x) are plotted in figure 18.5 for real x. For the 
smaller root, ¢2 = —v, however, the recurrence relation (18.78) becomes 


n(n—m)dy+aQ,-2=0 for n> 2, 


where m = 2v is now an even positive integer, i.e. m = 2,4,6,.... Starting with 
ao # 0 we may then calculate ap, a4,ao,..., but we see that when n = m the 
coefficient a, is formally infinite, and the method fails to produce a second 
solution in the form of a Frobenius series. 

In fact, by replacing v by —v in the definition of J,(x) given in (18.79), it can 
be shown that, for integer v, 


J_,(x) = (-1)"J, (x), 
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and hence that J,(x) and J_,(x) are linearly dependent. So, in this case, we cannot 
write the general solution to Bessel’s equation in the form (18.80). One therefore 
defines the function 


_ Jy(x) cos vt — J_y(x) 


Yy(x) (18.81) 


sin vm 
which is called a Bessel function of the second kind of order v (or, occasionally, 
a Weber or Neumann function). As Bessel’s equation is linear, Y,(x) is clearly a 
solution, since it is just the weighted sum of Bessel functions of the first kind. 
Furthermore, for non-integer v it is clear that Y,(x) is linearly independent of 
J,(x). It may also be shown that the Wronskian of J,(x) and Y,(x) is non-zero 
for all values of v. Hence J,(x) and Y,(x) always constitute a pair of independent 
solutions. 


>If n is an integer, show that Yn41/2(x) = (—1)"*1J_y-12(x). 


From (18.81), we have 


Inzt/2(x) cos(n + 5) — J-n—1/2(X) 


Y, x)= p 
+120) sin(n + 3)n 


If n is an integer, cos(n + 5)n = 0 and sin(n + 5)n = (—1)", and so we immediately obtain 
Yin41/2(%) = (—1)"*!J_,-1/2(x), as required. < 


The expression (18.81) becomes an indeterminate form 0/0 when vy is an 
integer, however. This is so because for integer v we have cosva = (—1)’ and 
J_,(x) = (—1)"J,(x). Nevertheless, this indeterminate form can be evaluated using 
YHO6pital’s rule (see chapter 4). Therefore, for integer v, we set 
J,y(x) cos ut — J_,(Xx) 


Y,(x) = lim 
[ov SIN M7 


: (18.82) 


which gives a linearly independent second solution for this case. Thus, we may 
write the general solution of Bessel’s equation, valid for all v, as 


y(x) = c1Jy(x) + cr ¥,(x). (18.83) 


The functions Yo(x), Y;(x) and Y2(x) are plotted in figure 18.6 

Finally, we note that, in some applications, it is convenient to work with 
complex linear combinations of Bessel functions of the first and second kinds 
given by 


HY(x) = J,(x) + iY, (x), HP (x) = J, (x) —i¥, (x); 


these are called, respectively, Hankel functions of the first and second kind of 
order v. 
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Figure 18.6 The first three integer-order Bessel functions of the second kind. 


18.5.3 Properties of Bessel functions J,(x) 


In physical applications, we often require that the solution is regular at x = 0, 
but, from its definition (18.81) or (18.82), it is clear that Y,(x) is singular at 
the origin, and so in such physical situations the coefficient cy in (18.83) must 
be set to zero; the solution is then simply some multiple of J,(x). These Bessel 
functions of the first kind have various useful properties that are worthy of 
further discussion. Unless otherwise stated, the results presented in this section 
apply to Bessel functions J,(x) of integer and non-integer order. 


Mutual orthogonality 


In section 17.4, we noted that Bessel’s equation (18.73) could be put into con- 
ventional Sturm-—Liouville form with p = x, q = —v?/x, 2 = o&? and p = x, 
provided «x is the argument of y. From the form of p, we see that there is no 
natural interval over which one would expect the solutions of Bessel’s equation 
corresponding to different eigenvalues / (but fixed v) to be automatically orthog- 
onal. Nevertheless, provided the Bessel functions satisfied appropriate boundary 
conditions, we would expect them to obey an orthogonality relationship over 
some interval [a,b] of the form 


b 
fi xJ\(ax)J\(Bx) dx = 0 fora # Bp. (18.84) 
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To determine the required boundary conditions for this result to hold, let us 
consider the functions f(x) = J,(ax) and g(x) = J,(Bx), which, as will be proved 
below, respectively satisfy the equations 


xf” + xf! + (0x? — v)f =0, (18.85) 
x2g" + xg" + (B2x? — v2)g = 0. (18.86) 


> Show that f(x) = J,(ax) satisfies (18.85). 


If f(x) = J,(ax) and we write w = ax, then 
df ddiw) Cf ,dJQw) 
a aw OM 
When these expressions are substituted into (18.85), its LHS becomes 
ad, #I,(w) dJ,(w) 
$242 
awe tap 


+ (0°? — v?)J,(w) 


@J, dJ,(w 
w (w) +w (w) + (w? — v7)J,(w). 
dw? dw 
But, from Bessel’s equation itself, this final expression is equal to zero, thus verifying that 
f(x) does satisfy (18.85). < 


Now multiplying (18.86) by f(x) and (18.85) by g(x) and subtracting them gives 


i 
ay ble! — f')] = (0? — B)xfe, (18.87) 
where we have used the fact that 
1 of af oy et p - of 
syle’ — af I] = xUfe" — 8f") + (fe! sf’). 


By integrating (18.87) over any given range x = a to x = b, we obtain 


b b 
J sfeosta dx =~ [xf eo) — xe 0] 


1 
2 pe 
which, on setting f(x) = J\(ax) and g(x) = J, (Bx), becomes 


b 
: xJy(ax)J,(Bx) dx = =— [Br (ax \l(Bx) ~ ax Bx (0x)] 
a 0 —p a 
(18.88) 


If « # B, and the interval [a,b] is such that the expression on the RHS of (18.88) 
equals zero, then we obtain the orthogonality condition (18.84). This happens, for 
example, if J,(ax) and J,(Bx) vanish at x = a and x = 5, or if J/(ax) and J/(Bx) 
vanish at x = a and x = b, or for many more general conditions. It should be 
noted that the boundary term is automatically zero at the point x = 0, as one 
might expect from the fact that the Sturm—Liouville form of Bessel’s equation 
has p(x) = x. 

If « = B, the RHS of (18.88) takes the indeterminant form 0/0. This may be 
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evaluated using I’HOpital’s rule, or alternatively we may calculate the relevant 
integral directly. 


> Evaluate the integral 


b 
i) J? (ax)x dx. 


Ignoring the integration limits for the moment, 


| Floxae = = J Flomau, 


where u = ax. Integrating by parts yields 


I = [ uyudu = west) — fh (u)J/(u)u du. 


Now Bessel’s equation (18.73) can be rearranged as 
wJ,(u) = v?J,(u) — uJ! (u) — J(u), 


which, on substitution into the expression for I, gives 


I= west) — fs (u)[v7J,(u) — uJ (u) — wJ""(u)] du 
a uJ, (u) — 3v7J;(u) + su? [Ji (u)]? te. 
Since u = «x, the required integral is given by 


b 


b 
| J?(ax)x dx = ; I(# - a) J(ax) + x? (ax)]?] (18.89) 


a 


which gives the normalisation condition for Bessel functions of the first kind. < 


Since the Bessel functions J,(x) possess the orthogonality property (18.88), we 
may expand any reasonable function f(x), i.e. one obeying the Dirichlet conditions 
discussed in chapter 12, in the interval 0 < x < b as a sum of Bessel functions of 
a given (non-negative) order v, 


f(xy= ss CnJy (nx), (18.90) 


n=0 


provided that the «, are chosen such that J,(%,b) = 0. The coefficients c, are then 
given by 


Cn = 


mean f F(X) Jv (tnx) x dx. (18.91) 


2: 
b Frail %n 


The interval is taken to be 0 < x < b, as then one need only ensure that the 
appropriate boundary condition is satisfied at x = b, since the boundary condition 
at x = 0 is met automatically. 
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> Prove the expression (18.91 ). 


If we multiply (18.90) by xJ,(a%x) and integrate from x = 0 to x = b then we obtain 


XJ (mx )dx = af XJ) (GmX) Jy (En) d 
J ioe: ) 


n=0 


b 
=o f J? (amx)x dx 
0 
= be, b? J" (Amb) =a 5m? Ie 4 1(Gnb), 


where in the last two lines we have used (18.88) with a, = « # B = a, (18.89), the fact 
that J,(%,b) = 0 and (18.95), which is proved below. < 


Recurrence relations 


The recurrence relations enjoyed by Bessel functions of the first kind, J,(x), can 
be derived directly from the power series definition (18.79). 


> Prove the recurrence relation 


d 
F(x" In(2)] = ¥°'r-10). (18.92) 


From the power series definition (18. is of J\(x) we obtain 
—1)"x 2v+2n 


> ene +n+1) 


d., 
aoe) 


(— 1)"x 2v+2n—1 
7 -» 2°+2n—InIT(y +n) 
: (—1)"x0—D+2 ; 
x" Jy_1(x). < 
” 3 I-A 1) pnb) ee) 


It may similarly be shown that 


d 
dx 
From (18.92) and (18.93) the remaining recurrence relations may be derived. 
Expanding out the derivative on the LHS of (18.92) and dividing through by 


x’—!, we obtain the relation 


[x "Jy (x)] = —x "Jy 41 (x). (18.93) 


XJ) (x) + vy (x) = xJy—1(x). (18.94) 


Similarly, by expanding out the derivative on the LHS of (18.93), and multiplying 
through by x’*t!, we find 


xJ1 (x) — v(x) = —xJy41(x). (18.95) 
Adding (18.94) and (18.95) and dividing through by x gives 
Jy 1x) = Jvg1 20) = 25,(%). (18.96) 
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Finally, subtracting (18.95) from (18.94) and dividing by x gives 


Jel) + Ios) = 0). (18.97) 


> Given that Jyj2(x) = (2/nx)!/? sinx and that J_1/2(x) = (2/nx)'/* cos x, express J3/2(x) 
and J_3/2(x) in terms of trigonometric functions. 


From (18.95) we have 


1 y 
J3/2(X) = 7 Dah) — Fj /2(x) 
1 2... 27 ie a ae 
=— |— sinx — {| — cosx + — | — sin x 
2x \ mx 1X 2x \ mx 
(ze) (Faxes) 
sinx —cosx]}. 
1X x 


Similarly, from (18.94) we have 


1 7 
J_32(x) = — 5,012) + JE (x) 


fl 4 \12 4 \12 1 2\12 
--3 (=) cosx- (=) sinx— (2) cos x 
2x \ mx 1X 2x \ mx 
(Fe) (ges-sn) 
cosx—sinx]. 
1X x 


We see that, by repeated use of these recurrence relations, all Bessel functions J,(x) of half- 
integer order may be expressed in terms of trigonometric functions. From their definition 
(18.81), Bessel functions of the second kind, Y,(x), of half-integer order can be similarly 
expressed. < 


Finally, we note that the relations (18.92) and (18.93) may be rewritten in 
integral form as 


[8 dred = 00, 
[stants dx = —x~" J, (x). 


If v is an integer, the recurrence relations of this section may be proved using 
the generating function for Bessel functions discussed below. It may be shown 
that Bessel functions of the second kind, Y,(x), also satisfy the recurrence relations 
derived above. 


Generating function 


The Bessel functions J,(x), where v = n is an integer, can be described by a 
generating function in a way similar to that discussed for Legendre polynomials 
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in subsection 18.1.2. The generating function for Bessel functions of integer order 
is given by 


G(x,h) = exp F (1- *)| = s Iy(xh. (18.98) 


n=—o 
By expanding the exponential as a power series, it is straightfoward to verify that 
the functions J;,(x) defined by (18.98) are indeed Bessel functions of the first kind, 
as given by (18.79). 
The generating function (18.98) is useful for finding, for Bessel functions of 
integer order, properties that can often be extended to the non-integer case. In 
particular, the Bessel function recurrence relations may be derived. 


b> Use the generating function to prove, for integer v, the recurrence relation (18.97), i.e. 


Jy-1(x) + Jygi(x) = So) 


Differentiating G(x, h) with respect to h we obtain 


6G(x,h) _ x 1 = nl 
7 ie (1 + a) Gx,A) = SO adooh, 


n=—H 


xo 


which can be written using (18.98) again as 


x 1 = nh n— 
. (1 +z) SS In(xyh” = > nJ,(x)e. 


n=—0o n=—0 


Equating coefficients of h” we obtain 
x 
2 [Jn(x) + Jns2(x)] = (n a 1)Jn4i(X), 
which, on replacing n by v — 1, gives the required recurrence relation. < 


Integral representations 


The generating function (18.98) is also useful for deriving integral representations 
of Bessel functions of integer order. 


> Show that for integer n the Bessel function J,(x) is given by 


Jn(x) = - | cos(nO — x sin 0) dé. 


By expanding out the cosine term in the integrand in (18.99) we obtain the integral 
1 T 
T= 5 | [cos(x sin 0) cos n@ + sin(x sin @) sin n@] dé. (18.100) 
0 


Now, we may express cos(x sin @) and sin(xsin@) in terms of Bessel functions by setting 
h = expi0 in (18.98) to give 


exp [Sexe if — exp(—i0))| = exp (ix sin @) = iS Jin(x) exp imé. 


m=—H 
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Using de Moivre’s theorem, expi@ = cos @ + isin 0, we then obtain 


exp (ix sin 0) = cos(x sin 0) + isin(x sin 0) = oy Im(x)(cos m@ + isin mé). 
Equating the real and imaginary parts of this expression gives 
cos(x sin 0) = > Jm(x) cos m0, 
sin(x sin @) = sz Jm(x) sin m0. 


m=—H 


Substituting these expressions into (18.100) then yields 


1 f* : : 
T= = a [ [Jin(x) cos m0 cos nO + J(x) sin m6 sin nO] dé. 


m=—a2 


However, using the orthogonality of the trigonometric functions [see equations (12.1)- 
(12.3)], we obtain 


T= =F [ulx) + Jnl)] = Jnl), 
which proves the integral representation (18.99). < 


Finally, we mention the special case of the integral representation (18.99) for 
n=0: 


T 2n 
Jo(x) = -{ cos(x sin 0) d0 = all cos(x sin 0) dO, 


since cos(x sin 0) repeats itself in the range 0 = z to 0 = 2x. However, sin(x sin 0) 
changes sign in this range and so 


1 2n 
= | sin(x sin 0) dO = 0. 
2n 0 
Using de Moivre’s theorem, we can therefore write 
1 2n 1 2n 
Jo(x) = x f exp(ix sin 0) d@ = = | exp(ix cos 0) dé. 
2n 0 2n 0 
There are in fact many other integral representations of Bessel functions; they 


can be derived from those given. 


18.6 Spherical Bessel functions 


When obtaining solutions of Helmholtz’ equation (V7 + k*)u = 0 in spherical 
polar coordinates (see section 21.3.2), one finds that, for solutions that are finite 
on the polar axis, the radial part R(r) of the solution must satisfy the equation 


rR" +2rR' + [k?r? —2(¢ + 1)JR = 0, (18.101) 
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where / is an integer. This equation looks very much like Bessel’s equation and 
can in fact be reduced to it by writing R(r) = r~!/7S(r), in which case S(r) then 
satisfies 


Ps" +75! + [kr — (¢+4)'] 8 =0. 


On making the change of variable x = kr and letting y(x) = S(kr), we obtain 


x’y" + xy +b? — E+ 5)'ly = 0, 


where the primes now denote d/dx. This is Bessel’s equation of order / + 5 
and has as its solutions y(x) = Jy41/2(x) and Y,/41/2(x). The general solution of 
(18.101) can therefore be written 


Rr) =r fed py 2lkr) + 2 Y412(kr)], 


where c; and co are constants that may be determined from the boundary 
conditions on the solution. In particular, for solutions that are finite at the origin 
we require c) = 0. 

The functions x7? Ty 44 /2(x) and x—!/2 Y/+1/2(x), when suitably normalised, are 
called spherical Bessel functions of the first and second kind, respectively, and are 
denoted as follows: 


jx) = [Ziennto, (18.102) 


n¢(x) = fz Yeneo. (18.103) 


For integer 7, we also note that Y/41/2(x) = (—1)/*'J__12(x), as discussed in 
section 18.5.2. Moreover, in section 18.5.1, we noted that Bessel functions of the 
first kind, J,(x), of half-integer order are expressible in closed form in terms of 
trigonometric functions. Thus, all spherical Bessel functions of both the first and 
second kinds may be expressed in such a form. In particular, using the results of 
the worked example in section 18.5.1, we find that 


jo) = —, (18.104) 
x 
n(x) = —. (18.105) 


Expressions for higher-order spherical Bessel functions are most easily obtained 
by using the recurrence relations for Bessel functions. 
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Show that the 7th spherical Bessel function is given by 


felx) = C1)°x? (; “= (18.106) 


where f/(x) denotes either j-(x) or n(x). 


The recurrence relation (18.93) for Bessel functions of the first kind reads 


d 
Jugal) = x" [0] 


Thus, on setting v = 7+ ; and rearranging, we find 


ya es 


x? Tr y3i2(x) x ae 7 


which on using (18.102) yields the recurrence relation 


d 
Jevilx) = —x’ — [x j(x)]. 


We now change / + 1 > / and iterate this ae 
jx) = 8 ay 09] 

_ (ap < {5 < [47-200] | 
= (—1)°x’ (< 2) noo 


This is the expression for j/(x) as given in (18.106). One may prove the result (18.106) for 


n/(x) in an analogous manner by setting v = 7 — 5 in the recurrence relation (18.92) for 


Bessel functions of the first kind and using the relationship Y/41/2(x) = (—1)1J_~-12(x). 


Using result (18.106) and the expressions (18.104) and (18.105), one quickly 
finds, for example, 


. sinx cosx ; 3 1 ae 3cosx 
AX) == - ; jx) = | = — = ) sinx — = 
x x x x x 
cosx  sinx 3 1 3 sinx 
n(x) = ——,- — —, n(x) = ; cos x a 
x x x x x 


Finally, we note that the orthogonality properties of the spherical Bessel functions 
follow directly from the orthogonality condition (18.88) for Bessel functions of 
the first kind. 


18.7 Laguerre functions 
Laguerre’s equation has the form 


xy” +(1—x)y’ + vy =0; (18.107) 
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it has a regular singularity at x = 0 and an essential singularity at x = oo. The 
parameter v is a given real number, although it nearly always takes an integer 
value in physical applications. The Laguerre equation appears in the description 
of the wavefunction of the hydrogen atom. Any solution of (18.107) is called a 
Laguerre function. 

Since the point x = 0 is a regular singularity, we may find at least one solution 
in the form of a Frobenius series (see section 16.3): 


Wx) = Same, (18.108) 
m=0 
Substituting this series into (18.107) and dividing through by x7!, we obtain 


5 lm + 0)(m to—1)+(1—x)\(m+o)+ vx] ayx” = 0. 
m=0 (18.109) 


Setting x = 0, so that only the m = 0 term remains, we obtain the indicial 
equation o? = 0, which trivially has o = 0 as its repeated root. Thus, Laguerre’s 
equation has only one solution of the form (18.108), and it, in fact, reduces to 
a simple power series. Substituting o = 0 into (18.109) and demanding that the 


coefficient of xt! vanishes, we obtain the recurrence relation 
m—v 
ans. = a nn: 
m+ (m 12 m 


As mentioned above, in nearly all physical applications, the parameter v takes 
integer values. Therefore, if v = n, where n is a non-negative integer, we see that 
An41 = An42 = +++: = 0, and so our solution to Laguerre’s equation is a polynomial 
of order n. It is conventional to choose dag = 1, so that the solution is given by 


—1)" 2 saa 
L(x) = ( 0) [ — ae a n vs ) xr? ladle (—1)"n! (18.110) 
n a! 
= m = au 
d| alam (18.111) 


where L,(x) is called the nth Laguerre polynomial. We note in particular that 
L,(0) = 1. The first few Laguerre polynomials are given by 


Lo(x) = 1, 3!L3(x) = —x3 + 9x? — 18x + 6, 


L(x) =—x +1, AlLa(x) = x4 — 16x? + 72x? — 96x + 24, 
Q'Lo(x) =x? —4x+2, 5!Ls(x) = —x° + 25x* — 200x3 + 600x? — 600x + 120. 


The functions Lo(x), Ly(x), Lo(x) and L3(x) are plotted in figure 18.7. 
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Ee ae 2 x 


Figure 18.7 The first four Laguerre polynomials. 


18.7.1 Properties of Laguerre polynomials 


The Laguerre polynomials and functions derived from them are important in 
the analysis of the quantum mechanical behaviour of some physical systems. We 
therefore briefly outline their useful properties in this section. 


Rodrigues’ formula 


The Laguerre polynomials can be expressed in terms of a Rodrigues’ formula 
given by 


Pages (xe), (18.112) 


~ nl dx" 


which may be proved straightforwardly by calculating the nth derivative explicitly 
using Leibnitz’ theorem and comparing the result with (18.111). This is illustrated 
in the following example. 
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» Prove that the expression (18.112) yields the nth Laguerre polynomial. 


Evaluating the nth derivative in (18.112) using Leibnitz’ theorem, we find 


e i. d’x" Qite 
Lil(x) Reo Ca 
= 


xr dxn-r 


e n! n! 


xa 1 beer an 
n! J ri(n—r)i(n—r)! Gy 


nr n! AF 
pe el rln—nin-nl * 


r=0 


Relabelling the summation using the index m = n —r, we obtain 


Ls Pe n! m 
L,(x) oa )) (mlp(n—m)! : 


m=0 


which is precisely the expression (18.111) for the nth Laguerre polynomial. < 


Mutual orthogonality 


In section 17.4, we noted that Laguerre’s equation could be put into Sturm— 
Liouville form with p = xe*, q =0, 2 =v and p = e™, and its natural interval 
is thus [0, co]. Since the Laguerre polynomials L,,(x) are solutions of the equation 
and are regular at the end-points, they must be mutually orthogonal over this 


x 


interval with respect to the weight function p = e™, ie. 


: Ly(x)Ly(x)e™* dx =0 ifn # k. 
0 


This result may also be proved directly using the Rodrigues’ formula (18.112). 
Indeed, the normalisation, when k = n, is most easily found using this method. 


> Show that 


Using the Rodrigues’ formula (18.112), we may write 


wo d" —1)" oa) a"Ly : 
I=— L(x) (x"e*) dx = (“) | —— x"e~* dx, 
0 IX nl Jo dx” 


where, in the second equality, we have integrated by parts n times and used the fact that the 


boundary terms all vanish. When d"L,,/dx" is evaluated using (18.111), only the derivative 
of the m =n term survives and that has the value [(—1)"n!n!]/[(n!)? 0!] = (—1)". Thus 


we have 
I -a/ xe ads = 
n! Jo 


where, in the second equality, we use the expression (18.153) defining the gamma function 
(see section 18.12). < 
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The above orthogonality and normalisation conditions allow us to expand any 
(reasonable) function in the interval 0 < x < oo in a series of the form 


a 


f(x) = ye an Ly(X), 


n=0 


in which the coefficients a, are given by 


= | ” f)En (ae dx. 


We note that it is sometimes convenient to define the orthonormal Laguerre func- 
tions ,(x) = e~*/?L,(x), which may also be used to produce a series expansion 
of a function in the interval 0 < x < 0. 


Generating function 
The generating function for the Laguerre polynomials is given by 


eo xh/(1-h) % ‘ 
6%)=—>- = S| Ly(x)h". (18.114) 
n=0 
We may prove this result by differentiating the generating function with respect to 
x and h, respectively, to obtain recurrence relations for the Laguerre polynomials, 
which may then be combined to show that the functions L,(x) in (18.114) do 
indeed satisfy Laguerre’s equation (as discussed in the next subsection). 


Recurrence relations 


The Laguerre polynomials obey a number of useful recurrence relations. The 
three most important relations are as follows: 


(n + LLng (x) = (2n + 1 — x)L, (x) — nLn_1 (x), (18.115) 
Ly-1(x) = L,,1(x) = L(x), (18.116) 
XL (x) = nLy(x) — nLy-1(x). (18.117) 


The first two relations are easily derived from the generating function (18.114), 
and may be combined straightforwardly to yield the third result. 


> Derive the recurrence relations (18.115) and (18.116). 


Differentiating the generating function (18.114) with respect to h, we find 
eG (lL—x- hye/0—) 
oh (1 —h) 


a nL,h". 


Thus, we may write 


(l1—x—h) Dy L,h" = (1—hy ay nL,h"!, 
and, on equating coefficients of h" on each side, we obtain 
(1 =: x)En —Lhei = (n at 1b = 2nLy + (n = 1)Ly-1, 
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which trivially rearranges to give the recurrence relation (18.115). 
To obtain the recurrence relation (18.116), we begin by differentiating the generating 
function (18.114) with respect to x, which yields 


6G _he —xh/(1—h) 
ane EE h", 
Ox “dhe Boy 
and thus we have 
—h > Lh" =(1—h) > Lin". 
Equating coefficients of h” on each side then gives 
—Ey= nib =U =, Eee 1 


which immediately simplifies to give (18.116). < 


18.8 Associated Laguerre functions 


The associated Laguerre equation has the form 
xy” +(m+1—x)y’+ny =0; (18.118) 


it has a regular singularity at x = 0 and an essential singularity at x = «0. We 
restrict our attention to the situation in which the parameters n and m are both 
non-negative integers, as is the case in nearly all physical problems. The associated 
Laguerre equation occurs most frequently in quantum-mechanical applications. 
Any solution of (18.118) is called an associated Laguerre function. 

Solutions of (18.118) for non-negative integers n and m are given by the 
associated Laguerre polynomials 


im 


d 
Ly (x) = (—1)" 5 Ensm(X), (18.119) 


where L,,(x) are the ordinary Laguerre polynomials$ 


> Show that the functions L)"(x) defined in (18.119) are solutions of (18.118). 


Since the Laguerre polynomials L,(x) are solutions of Laguerre’s equation (18.107), we 
have 


XL iam +(1- x) Li gm +(n+ m)Ln4m = 0. 
Differentiating this equation m times using Leibnitz’ theorem and rearranging, we find 
XL + (+L x)Litn) + ML = O. 


On multiplying through by (—1)” and setting Li? = (10. in accord with (18.119), 
we obtain 


x(Li")" + (m+ 1 —x)(L")’ + nL” = 0, 


which shows that the functions L)" are indeed solutions of (18.118). < 


S Note that some authors define the associated Laguerre polynomials as £""(x) = (d"/dx")Ln(x), 
which is thus related to our expression (18.119) by Li'(x) = (—L)" £1. ,(x)- 
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In particular, we note that L°(x) = L,,(x). As discussed in the previous section, 
L,(x) is a polynomial of order n and so it follows that L'"(x) is also. The first few 
associated Laguerre polynomials are easily found using (18.119): 


Lo (x) = 1, 
Li (x) = —x+m-+1, 
Q'L2'(x) = x? — 2(m + 2)x + (m + 1)(m + 2), 
3IL™(x) = —x3 + 3(m + 3)x? — 3(m + 2)(m + 3)x + (m + 1)(m + 2)(m 4 3). 


Indeed, in the general case, one may show straightforwardly, from the definition 
(18.119) and the expression (18.111) for the ordinary Laguerre polynomials, that 


Mr.) _ . k (n+ m)! k 
L(x) DI es emesis ade (18.120) 


18.8.1 Properties of associated Laguerre polynomials 


The properties of the associated Laguerre polynomials follow directly from those 
of the ordinary Laguerre polynomials through the definition (18.119). We shall 
therefore only briefly outline the most useful results here. 


Rodrigues’ formula 


A Rodrigues’ formula for the associated Laguerre polynomials is given by 


ex” qd" = 
Te (18.121) 


Li(x) = 


It can be proved by evaluating the nth derivative using Leibnitz’ theorem (see 
exercise 18.7). 


Mutual orthogonality 


In section 17.4, we noted that the associated Laguerre equation could be trans- 
formed into a Sturm—Liouville one with p = x""*'e-*, q = 0,2 =n and p = x"e*, 
and its natural interval is thus [0,00]. Since the associated Laguerre polynomials 
L(x) are solutions of the equation and are regular at the end-points, those 
with the same m but differing values of the eigenvalue 2 = n must be mutually 
orthogonal over this interval with respect to the weight function p = x"e*, Le. 


[ encancatedx=0 ifn ék. 
0 


This result may also be proved directly using the Rodrigues’ formula (18.121), as 
may the normalisation condition when k =n. 
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> Show that 


(n+m)! 
n! 


I =| L(x) Li (x)x"e~* dx = 
0 


Using the Rodrigues’ formula (18.121), we may write 
1 io) d" —1 n co) d’?Lm 
f= =| L(x )— (x"t™e-*) dx = ( ) | nada Us xm ex dx, 
n! Jo dx" n! Jo dx" 
where, in the second equality, we have integrated by parts n times and used the fact that 
the boundary terms all vanish. From (18.120) we see that d"L"/dx" = (—1)". Thus we have 


1 [* ! 
PhS xt ox dx = (n Bg m) 


n! Jo n! 


bs 


where, in the second equality, we use the expression (18.153) defining the gamma function 
(see section 18.12). < 


The above orthogonality and normalisation conditions allow us to expand any 
(reasonable) function in the interval 0 < x < « in a series of the form 


a3 Un L(x 


n=0 


in which the coefficients a, are given by 


dh = arom | f(X)LM(x)xe™ dx. 


We note that it is sometimes convenient to define the orthogonal associated 
Laguerre functions 6"(x) = x""/2e~*/?L™(x), which may also be used to produce a 
series expansion of a function in the interval 0 < x < ©. 


Generating function 
The generating function for the associated Laguerre polynomials is given by 


e7xh/(1-h) ® 
G(x, h) = ———_ = Li(x)h" (18.123) 
el —hyntl d 
This can be obtained by differentiating the generating function (18.114) for the 
ordinary Laguerre polynomials m times with respect to x, and using (18.119). 


> Use the generating function (18.123) to obtain an expression for L'"(0). 


From (18.123), we have 


n 1 
yu (O)h" = > —hyn+t 


n=0 


14 ne Wnt Dt 2) po a. gp MAN (m+n) 


ope 7 a 
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where, in the second equality, we have expanded the RHS using the binomial theorem. On 
equating coefficients of h", we immediately obtain 


(n+m)! 
nim! —~ 


Li(0) = 


Recurrence relations 


The various recurrence relations satisfied by the associated Laguerre polynomials 
may be derived by differentiating the generating function (18.123) with respect to 
either or both of x and h, or by differentiating with respect to x the recurrence 
relations obeyed by the ordinary Laguerre polynomials, discussed in section 18.7.1. 
Of the many recurrence relations satisfied by the associated Laguerre polynomials, 
two of the most useful are as follows: 


(n+ 1)L", (x) = (Qn +m+1—x)L™(x)— (n+ m)L™,(x), (18.124) 
x(Li')'(x) = nLii(x) — (n + m)L 7 4 (x). (18.125) 


For proofs of these relations the reader is referred to exercise 18.7. 


18.9 Hermite functions 
Hermite’s equation has the form 
y" —2xy' + 2vy =0, (18.126) 


and has an essential singularity at x = oo. The parameter v is a given real 
number, although it nearly always takes an integer value in physical applications. 
The Hermite equation appears in the description of the wavefunction of the 
harmonic oscillator. Any solution of (18.126) is called a Hermite function. 

Since x = 0 is an ordinary point of the equation, we may find two linearly 
independent solutions in the form of a power series (see section 16.2): 


yx) = So mx”. (18.127) 
m=0 


Substituting this series into (18.107) yields 

oO 

SS [Gm + 10m + 2)dmy2 + 2(v — m)am] x” = 0. 

m=0 
Demanding that the coefficient of each power of x vanishes, we obtain the 
recurrence relation 

____2(v—m) 
Fe Gee GeE a) 
As mentioned above, in nearly all physical applications, the parameter v takes 

integer values. Therefore, if vy = n, where n is a non-negative integer, we see that 


624 


18.9 HERMITE FUNCTIONS 


Figure 18.8 The first four Hermite polynomials. 


An42 = An44 = ++: =0, and so one solution of Hermite’s equation is a polynomial 
of order n. For even n, it is conventional to choose ay = (—1)"/?n!/(n/2)!, whereas 
for odd n one takes a, = (—1)""92n1/[5(n— 1)]!. These choices allow a general 
solution to be written as 


n(n — 1)(n — 2)(n— 3) 


7 (2x)"4 — - - (18.128) 


H,(x) = (2x)" — n(n — 1)(2x)""! + 
[n/2] 


m nl —2m 
= Ce mln amie” ie (18.129) 


where H,,(x) is called the nth Hermite polynomial and the notation [n/2] denotes 
the integer part of n/2. We note in particular that H,(—x) = (—1)"H,(x). The 
first few Hermite polynomials are given by 


Ao(x) = 1, H3(x) = 8x? — 12x, 

Ay (x) = 2x, H4(x) = 16x4* — 48x? + 12, 

Ho(x) = 4x? — 2, Hs(x) = 32x — 160x3 + 120x. 
The functions Ho(x), H1(x), H2(x) and H3(x) are plotted in figure 18.8. 
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18.9.1 Properties of Hermite polynomials 


The Hermite polynomials and functions derived from them are important in the 
analysis of the quantum mechanical behaviour of some physical systems. We 
therefore briefly outline their useful properties in this section. 


Rodrigues’ formula 


The Rodrigues’ formula for the Hermite polynomials is given by 


H,(x) = (rye? “e-*). (18.130) 
dx" 


This can be proved using Leibnitz’ theorem. 


> Prove the Rodrigues’ formula (18.130) for the Hermite polynomials. 


2 
— x2 


Letting u = e 


and differentiating with respect to x, we quickly find that 
ul’ + 2xu = 0. 
Differentiating this equation n+ 1 times using Leibnitz’ theorem then gives 
u"?) 4 xu") 4. 2(n + Lu” =0, 
which, on introducing the new variable v = (—1)"u", reduces to 
v" + 2xv' + 2(n + 1)v = 0. (18.131) 


Now letting y = ee, we may write the derivatives of v as 
of =e" (y! — 2xy), 
" 


v= ee (y" —4xy' + 4x?y — 2y). 


Substituting these expressions into (18.131), and dividing through by eo, finally yields 


Hermite’s equation, 

y” — 2xy + 2ny = 0, 
thus demonstrating that y = (—1)"e" d"(e™”) /dx" is indeed a solution. Moreover, since 
this solution is clearly a polynomial of order n, it must be some multiple of H,(x). The 


normalisation is easily checked by noting that, from (18.130), the highest-order term is 
(2x)", which agrees with the expression (18.128). < 


Mutual orthogonality 


We saw in section 17.4 that Hermite’s equation could be cast in Sturm—Liouville 
form with p = e™, q =0, 2 = 2n and p = e-*’, and its natural interval is thus 
[—co, 0]. Since the Hermite polynomials H,,(x) are solutions of the equation and 
are regular at the end-points, they must be mutually orthogonal over this interval 
with respect to the weight function p = e™, ie. 

ioe) 


H,(x)Hy(xje™ dx =0 ifn #k. 


This result may also be proved directly using the Rodrigues’ formula (18.130). 
Indeed, the normalisation, when k = n, is most easily found in this way. 
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> Show that 


l= / H,(x)H,(x)e~ dx = 2"n! Ja. 


Using the Rodrigues’ formula (18.130), we may write 


ioe) a" H. 3 
A, x x= —e™* ab > 
-y fr (x) — - ale )dx ae e x 


where, in the second equality, we have integrated by parts n times and used the fact that 
the boundary terms all vanish. From (18.128) we see that d"H,,/dx" = 2"n!. Thus we have 


is 2int f em dx = nl, 


where, in the second equality, we use the standard result for the area under a Gaussian 
(see section 6.4.2). < 


The above orthogonality and normalisation conditions allow any (reasonable) 
function in the interval —coo < x < © to be expanded in a series of the form 


f(x) = y a, H,(x), 


in which the coefficients a, are given by 


ee) 


1 
21! [te Joo 


We note that it is sometimes convenient to define the orthogonal Hermite functions 
brl(x) = e-*/2H,(x); they also may be used to produce a series expansion of a 
function in the interval —coo < x < o. Indeed, ¢,(x) is proportional to the 
wavefunction of a particle in the nth energy level of a quantum harmonic 
oscillator. 


Gy = f(x)Hnlxje™ dx. 


Generating function 
The generating function equation for the Hermite polynomials reads 


Cee" = yo F(X) jn (18.133) 
n=0 


a result that may be proved using the Rodrigues’ formula (18.130). 


> Show that the functions H,(x) in (18.133) are the Hermite polynomials. 


It is often more convenient to write the generating function (18.133) as 


2 ‘- x A, 
G(x,h) =e eh? = s: Hil) pn 


n=0 
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Differentiating this form k times with respect to h gives 


a k 


k ak 
2a Hi, p-k aG es 0 ooh? ( 1ker a" ene h? 

; (n—k)! ohk ohk Oxk : 
n=kK 


Relabelling the summation on the LHS using the new index m = n—k, we obtain 


al 


An+k m kx? ok —(x—h)? 
or = Cie axe 


m=0 


Setting h = 0 in this equation, we find 
Hy(x) = (-1fe* —(e™ ), 
which is the Rodrigues’ formula (18.130) for the Hermite polynomials. < 
The generating function (18.133) is also useful for determining special values 


of the Hermite polynomials. In particular, it is straightforward to show that 
H,(0) = (—1)"(2n)!/n! and Hy, 41(0) = 0. 


Recurrence relations 


The two most useful recurrence relations satisfied by the Hermite polynomials 
are given by 


Ay y1(x) = 2xH,(x) — 2nHy_1(x), (18.134) 
A(x) = 2nH,1(x). (18.135) 


The first relation provides a simple iterative way of evaluating the nth Hermite 
polynomials at some point x = xo, given the values of Ho(x) and Hj(x) at that 
point. For proofs of these recurrence relations, see exercise 18.5. 


18.10 Hypergeometric functions 
The hypergeometric equation has the form 
x(1—x)y” + [e— (a+b + 1)x]y’ — aby = 0, (18.136) 


and has three regular singular points, at x = 0,1, 0, but no essential singularities. 
The parameters a, b and c are given real numbers. 

In our discussions of Legendre functions, associated Legendre functions and 
Chebyshev functions in sections 18.1, 18.2 and 18.4, respectively, it was noted that 
in each case the corresponding second-order differential equation had three regular 
singular points, at x = —1, 1,00, and no essential singularities. The hypergeometric 
equation can, in fact, be considered as the ‘canonical form’ for second-order 
differential equations with this number of singularities. It may be shown’ that, 


S See, for example, J. Mathews and R. L. Walker, Mathematical Methods of Physics, 2nd edn (Reading 
MA: Addision—Wesley, 1971). 


628 


18.10 HYPERGEOMETRIC FUNCTIONS 


by making appropriate changes of the independent and dependent variables, 
any second-order differential equation with three regular singularities and an 
ordinary point at infinity can be transformed into the hypergeometric equation 
(18.136) with the singularities at = —1, 1 and «. As we discuss below, this allows 
Legendre functions, associated Legendre functions and Chebyshev functions, for 
example, to be written as particular cases of hypergeometric functions, which are 
the solutions to (18.136). 

Since the point x = 0 is a regular singularity of (18.136), we may find at least 
one solution in the form of a Frobenius series (see section 16.3): 


a 


yx) = So anx™?, (18.137) 


n=0 


Substituting this series into (18.136) and dividing through by x7~!, we obtain 


sa x\(n+oa)n+o—1)+ [e—(at+ b+ 1)x](n + o) — abx} a,x" = 0. 
= (18.138) 


Setting x = 0, so that only the n = 0 term remains, we obtain the indicial equation 
o(o — 1) +co = 0, which has the roots o = 0 and o = 1 —c. Thus, provided 
c is not an integer, one can obtain two linearly independent solutions of the 
hypergeometric equation in the form (18.137). 

For o = 0 the corresponding solution is a simple power series. Substituting 
o = 0 into (18.138) and demanding that the coefficient of x” vanishes, we find the 
recurrence relation 


n{(n — 1) + cla, — [(n— 1)(a+b+n—1)+ab]a,_; = 0, 
(18.139) 


which, on simplifying and replacing n by n + 1, yields the recurrence relation 


i SE (18.140) 
+n 


It is conventional to make the simple choice aj = 1. Thus, provided c is not a 
negative integer or zero, we may write the solution as follows: 


abx  ala+1)b(b +1) x? 


F(a,b,c;x) =1+ Th ee 1) 7 (18.141) 
= Te) T(a < an (b +n) x’ 
= Tar® s ren aD (18.142) 


where F(a,b,c;x) is known as the hypergeometric function or hypergeometric 
series, and in the second equality we have used the property (18.154) of the 
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gamma functions It is straightforward to show that the hypergeometric series 
converges in the range |x| < 1. It also converges at x = 1 if c > a+b and 
at x = —-1 ifc > a+b—1. We also note that F(a,b,c;x) is symmetric in the 
parameters a and b, i.e. F(a,b,c;x) = F(b, a,c; x). 

The hypergeometric function y(x) = F(a,b,c;x) is clearly not the general 
solution to the hypergeometric equation (18.136), since we must also consider the 
second root of the indicial equation. Substituting o = 1 —c into (18.138) and 
demanding that the coefficient of x" vanishes, we find that we must have 


n(n + 1—c)a, — [(n—c)\a+b+n—c)+ abla, =0, 


which, on comparing with (18.139) and replacing n by n+ 1, yields the recurrence 
relation 


(a—c+1+4+n)\(b—c+1+n) 
(n+ 1)(2—c+n) 

We see that this recurrence relation has the same form as (18.140) if one makes 

the replacements a> a—c+1,b—>b—c+1 and c— 2—c. Thus, provided c, 

a—b and c—a—b are all non-integers, the general solution to the hypergeometric 

equation, valid for |x| < 1, may be written as 


aH = an. 


y(x) = AF(a, b,c; x) + Bx! *F(a—c+1,b—¢c+1,2—c¢;x), 
(18.143) 


where A and B are arbitrary constants to be fixed by the boundary conditions on 
the solution. If the solution is to be regular at x = 0, one requires B = 0. 


18.10.1 Properties of hypergeometric functions 


Since the hypergeometric equation is so general in nature, it is not feasible to 
present a comprehensive account of the hypergeometric functions. Nevertheless, 
we outline here some of their most important properties. 


Special cases 


As mentioned above, the general nature of the hypergeometric equation allows us 
to write a large number of elementary functions in terms of the hypergeometric 
functions F(a, b,c;x). Such identifications can be made from the series expansion 
(18.142) directly, or by transformation of the hypergeometric equation into a more 
familiar equation, the solutions to which are already known. Some particular 
examples of well known special cases of the hypergeometric function are as 
follows: 


S We note that it is also common to denote the hypergeometric function by F\(a,b,c;x). This 
slightly odd-looking notation is meant to signify that, in the coefficient of each power of x, there 
are two parameters (a and 5) in the numerator and one parameter (c) in the denominator. 
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F(a,b,b:x) = (1 —x)~4, F(5,4,3;x°) =x"! sin! x, 

F(1,1,2;—x) = x7! In(1 + x), F(4,1,3;—x?) = x7! tan x, 

Jim F(1,m,1;x/m) = e*, F(5, 1,35x7) = 3x7! Inf(1 + x)/(1—x)], 
F(5, 3.55 : sin’ x) = cos x, F(m+ 1,—m, 1;(1 — x)/2) = Pin(x), 
F(5,p p, p;sin? x) = sec x, F(m,—m, $3 — x)/2) = T(x), 


where m is an integer, P,,(x) is the mth Legendre polynomial and T,,,(x) is the 
mth Chebyshev polynomial of the first kind. Some of these results are proved in 
exercise 18.11. 


> Show that F(m,—m, nel — x)/2) = T(x). 

Let us prove this result by transforming the hypergeometric equation. The form of the 
result suggests that we should make the substitution x = (1 — z)/2 into (18.136), in which 
case d/dx = —2d/dz. Thus, letting u(z) = y(x) and setting a = m, b m and c = 1/2, 
(18.136) becomes 


(1—z) (1 +z) ,Cu 1 1—z du 
5) 5) (—2) im + |5—(m—m-+ 1) 5) ( VE (m)(—m)u = 0. 
On simplifying, we obtain 
@u du, 
(20-45 za +mu=0, 


which has the form of Chebyshev’s equation, (18.54). This equation has u(z) = T,,(z) a: 
its power series solution, and so F(m,—m, 5 $3( (1 — z)/2) and T,,(z) are equal to thie a 
normalisation factor. On comparing the expressions (18.141) and (18.56) at x = 0, ie. at 
z = 1, we see that they both have value 1. Hence, the normalisations already agree and 
we obtain the required result. < 


Integral representation 


One of the most useful representations for the hypergeometric functions is in 
terms of an integral, which may be derived using the properties of the gamma 
and beta functions discussed in section 18.12. The integral representation reads 


Tc b c—b-1 a 
= rarern f toda 


(18.144) 


and requires c > b > 0 for the integral to converge. 


> Prove the result (18.144). 


From the series expansion mex we have 


. T(at+nr(b +n) x" 
F(a,b,c;x) ait ae T(c +n) n! 


a” T(c) = x 
a Tororesh > T(a+n)B(b+n,c— b> 
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where in the second equality we have used the expression (18.165) relating the gamma and 
beta functions. Using the definition (18.162) of the beta function, we then find 


T(c) = x fl b+n-1 c—b-1 
F(a,b,c;x) = rareresy Ota, orl — 2) dt 


T(c) b-1 c—b-1 T(a +n) (tx)" 
= Tere res f a a SS Tia) nl? 


n=0 


where in the second equality we have rearranged the expression and reversed the order 
of integration and summation. Finally, one recognises the sum over n as being equal to 
(1 — tx)~“, and so we obtain the final result (18.144). < 


The integral representation may be used to prove a wide variety of properties of 
the hypergeometric functions. As a simple example, on setting x = 1 in (18.144), 
and using properties of the beta function discussed in section 18.12.2, one quickly 
finds that, provided c is not a negative integer or zero and c > a+b, 


T(c)'(c —a—b) 


PORE Tes anent): 


Relationships between hypergeometric functions 


There exist a great many relationships between hypergeometric functions with 
different arguments. These are most easily derived by making use of the integral 
representation (18.144) or the series form (18.141). It is not feasible to list all the 
relationships here, so we simply note two useful examples, which read 


F(a,b,c;x) = (1—x)* F(c — a,c — b,c; x), (18.145) 
F'(a,b,c;x) = Ce grb acag (18.146) 
Cc 


where the prime in the second relation denotes d/dx. The first result follows 
straightforwardly from the integral representation using the substitution t = 
(1 — u)/(1 — ux), whereas the second result may be proved more easily from the 
series expansion. 

In addition to the above results, one may also derive relationships between 
F(a,b,c;x) and any two of the six ‘contiguous functions’ F(a +1,b,c;x), F(a,b+ 
1,c;x) and F(a,b,c +1;x). These ‘contiguous relations’ serve as the recurrence 
relations for the hypergeometric functions. An example of such a relationship is 


(c—a)F(a—1,b,c;x)+(2a—c—ax + bx)F(a, b,c; x) +.a(x— 1)F(a+1,b,c;x) = 


Repeated application of such relationships allows one to express F(a+l,b+m,c+ 
n;x), where I, m,n are integers (with c+n not equalling a negative integer or zero), 
as a linear combination of F(a,b,c;x) and one of its contiguous functions. 
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18.11 Confluent hypergeometric functions 
The confluent hypergeometric equation has the form 
xy” +(ce—x)y’ —ay =0; (18.147) 


it has a regular singularity at x = 0 and an essential singularity at x = o. 
This equation can be obtained by merging two of the singularities of the ordinary 
hypergeometric equation (18.136). The parameters a and c are given real numbers. 


> Show that setting x = z/b in the hypergeometric equation, and letting b > «, yields the 
confluent hypergeometric equation. 


Substituting x = z/b into (18.136), with d/dx = bd/dz, and letting u(z) = y(x), we obtain 

2. 

bz (1- =) a + [be—(atb+ tye) — abu =0, 

which clearly has regular singular points at z = 0, b and o. If we now merge the last two 
singularities by letting b > o0, we obtain 

zu" +(c—z)u' —au=0, 
where the primes denote d/dz. Hence u(z) must satisfy the confluent hypergeometric 
equation. < 

In our discussion of Bessel, Laguerre and associated Laguerre functions, it was 
noted that the corresponding second-order differential equation in each case had a 
single regular singular point at x = 0 and an essential singularity at x = oo. From 
table 16.1, we see that this is also true for the confluent hypergeometric equation. 
Indeed, this equation can be considered as the ‘canonical form’ for second- 
order differential equations with this pattern of singularities. Consequently, as we 
mention below, the Bessel, Laguerre and associated Laguerre functions can all be 
written in terms of the confluent hypergeometric functions, which are the solutions 
of (18.147). 

The solutions of the confluent hypergeometric equation are obtained from 
those of the ordinary hypergeometric equation by again letting x — x/b and 
carrying out the limiting process b > oo. Thus, from (18.141) and (18.143), two 
linearly independent solutions of (18.147) are (when c is not an integer) 


ax  a(a+1)z? 
cl! c(c+1)2! 
yo(x) = x!-°M(a—e + 1,2 —c;x), (18.149) 


+++ = M(a,c;x), (18.148) 


where M(a,c;x) is called the confluent hypergeometric function (or Kummer 
function)§ It is worth noting, however, that y;(x) is singular when c = 0,—1,—2,... 
and y2(x) is singular when c = 2,3,4,.... Thus, it is conventional to take the 


S We note that an alternative notation for the confluent hypergeometric function is | Fj(a,c;x). 


633 


SPECIAL FUNCTIONS 


second solution to (18.147) as a linear combination of (18.148) and (18.149) given 
by 

1 M(a,c3;x) 1--M(a—c + 1,2 —¢;x) 
sinne |T(a—c+IIQ Ti@re—o) 


U(a,c;x) = 


This has a well behaved limit as c approaches an integer. 


18.11.1 Properties of confluent hypergeometric functions 


The properties of confluent hypergeometric functions can be derived from those 
of ordinary hypergeometric functions by letting x — x/b and taking the limit 
b > o, in the same way as both the equation and its solution were derived. A 
general procedure of this sort is called a confluence process. 


Special cases 


The general nature of the confluent hypergeometric equation allows one to write 
a large number of elementary functions in terms of the confluent hypergeometric 
functions M(a,c;x). Once again, such identifications can be made from the series 
expansion (18.148) directly, or by transformation of the confluent hypergeometric 
equation into a more familiar equation for which the solutions are already 
known. Some particular examples of well known special cases of the confluent 
hypergeometric function are as follows: 


X es a 
M(a,a;x) =e, Mag Se 
Im! 
M(-n, 13x) = L(x), M(—n,m+1;x) = emilee 
—1)"n! (—1)"n! Hanyi(x) 
M(— eee ee H Ri Ee 
( n, 35x ) (2n)! on(X), ( n, 55x ) 2(2n + 1! a > 


x), M(5.35-x°) = YF erf(x), 


where n and m are integers, L’"(x) is an associated Legendre polynomial, H,,(x) is 
a Hermite polynomial, J,(x) is a Bessel function and erf(x) is the error function 


discussed in section 18.12.4. 


M(v + $,2v + 1;2ix) = vle™( 


Integral representation 


Using the integral representation (18.144) of the ordinary hypergeometric func- 
tion, exchanging a and b and carrying out the process of confluence gives 


T(c) ; tx ,a—1 c—a-1 
M(a, c, x) = == ———~ e*t* “(1 —2t) dt, 
T(ar(c— a) | (18.150) 
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which converges provided c > a > 0. 


> Prove the result (18.150). 


Since F(a, b,c; x) is unchanged by swapping a and b, we may write its integral representation 
(18.144) as 


T(c) ; 1 a1 ~b 
F(a,b,c;x) = ————_ “(dt (1. dt. 
(abe) = pees [ta —-m den 
Setting x = z/b and taking the limit b > o, we obtain 
. = Tc) ' a—1 c—a-1 ]: tz me 
MUG D2) = Tread : “(1 —-t) jim La dt. 


Since the limit is equal to e’”, we obtain result (18.150). < 


Relationships between confluent hypergeometric functions 


A large number of relationships exist between confluent hypergeometric functions 
with different arguments. These are straightforwardly derived using the integral 
representation (18.150) or the series form (18.148). Here, we simply note two 
useful examples, which read 


M(a,c;x) = e*M(c—a,c;—x), (18.151) 
M'(a,c;x) = “M(a +1,¢ + 1;x), (18.152) 
Cc 


where the prime in the second relation denotes d/dx. The first result follows 
straightforwardly from the integral representation, and the second result may be 
proved from the series expansion (see exercise 18.19). 

In an analogous manner to that used for the ordinary hypergeometric func- 
tions, one may also derive relationships between M(a,c;x) and any two of the 
four ‘contiguous functions’ M(a + 1,c;x) and M(a,c+1;x). These serve as the 
recurrence relations for the confluent hypergeometric functions. An example of 
such a relationship is 


(c — a)M(a—1,c;x) + (2a—c+x)M(a,c;x) —aM(a+1,c;x) =0. 


18.12 The gamma function and related functions 


Many times in this chapter, and often throughout the rest of the book, we have 
made mention of the gamma function and related functions such as the beta and 
error functions. Although not derived as the solutions of important second-order 
ODEs, these convenient functions appear in a number of contexts, and so here 
we gather together some of their properties. This final section should be regarded 
merely as a reference containing some useful relations obeyed by these functions; 
a minimum of formal proofs is given. 
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18.12.1 The gamma function 
The gamma function T(n) is defined by 


Tn) = iD x le dy, (18.153) 
0 


which converges for n > 0, where in general n is a real number. Replacing n by 
n+ 1 in (18.153) and integrating the RHS by parts, we find 


T(n+ 1) -| x"e* dx 
0 


% 
= n | x" Te dx, 
0 


from which we obtain the important result 
T(n+ 1) =nI(n). (18.154) 
From (18.153), we see that I’(1) = 1, and so, if n is a positive integer, 
T(n+ 1) =n!. (18.155) 


In fact, equation (18.155) serves as a definition of the factorial function even for 

non-integer n. For negative n the factorial function is defined by 
(n+ m)! 

(nt m\(n+m—1)---(n+1) 


where m is any positive integer that makes n+ m > 0. Different choices of m 


= (18.156) 


(> —n) do not lead to different values for n!. A plot of the gamma function is 
given in figure 18.9, where it can be seen that the function is infinite for negative 
integer values of n, in accordance with (18.156). For an extension of the factorial 
function to complex arguments, see exercise 18.15. 

By letting x = y? in (18.153), we immediately obtain another useful represen- 
tation of the gamma function given by 


rae | pte ay, (18.157) 
0 


Setting n = 5 we find the result 
I (5) =2 [ ePay= f ev dy= Jt, 
0 —o 
where have used the standard integral discussed in section 6.4.2. From this result, 


T(n) for half-integral n can be found using (18.154). Some immediately derivable 
factorial values of half integers are 


(—3)!=—2Vm (-a)!=Vm (G)!=3V% (3)! = aN 
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Figure 18.9 The gamma function I'(n). 


Moreover, it may be shown for non-integral n that the gamma function satisfies 
the important identity 
1 


T(n)P(1 —n) = (18.158) 


sinnt 
This is proved for a restricted range of n in the next section, once the beta 
function has been introduced. 

It can also be shown that the gamma function is given by 


1 1 139 
T(n+1) = V2an n"e" (1+ + 7 7 7) =n!, 
12n = _.288n 51 840n (18.159) 


which is known as Stirling’s asymptotic series. For large n the first term dominates, 
and so 


n! = /2nn n"e"; (18.160) 


this is known as Stirling’s approximation. This approximation is particularly useful 
in statistical thermodynamics, when arrangements of a large number of particles 
are to be considered. 


> Prove Stirling’s approximation n! ~ ,/21n n"e" for large n. 


From (18.153), the extended definition of the factorial function (which is valid for n > —1) 
is given by 


n= [verax= [ erin dy, (18.161) 
0 0 
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If we let x =n+y, then 


Inx =Inn+In (1+ ~) 


=Inn+--=54+-5-:::. 


Substituting this result into (18.161), we obtain 


(oe) y y? 
i ae , 
n! is exp [ (inn+ a ae + ) n q dy. 


Thus, when n is sufficiently large, we may approximate n! by 


) 
= —y? pe a 
nix efinn . e y~ /(2n) dy = enlnn n [2xn a2 Ann n'e H 


0 


which is Stirling’s approximation (18.160). < 


18.12.2 The beta function 


The beta function is defined by 
1 
B(m,n) = ih x" — xy"! dx, (18.162) 
0 


which converges for m > 0, n > 0, where m and n are, in general, real numbers. 
By letting x = 1 — y in (18.162) it is easy to show that B(m,n) = B(n,m). Other 
useful representations of the beta function may be obtained by suitable changes 
of variable. For example, putting x = (1 + y)~! in (18.162), we find that 


ive) yr dy 
B(m,n) = ————.. 18.163 
( ) | (i +yynen ( ) 


Alternatively, if we let x = sin’ 0 in (18.162), we obtain immediately 


n/2 
B(m,n) = 2 i] sin”! 0 cos”! 6 dO. (18.164) 
0 
The beta function may also be written in terms of the gamma function as 
T(m)I(n) 
B = 18.165 
"= Fm (18.165) 


> Prove the result (18.165). 


Using (18.157), we have 


T(a)T'(m) = 4 on le ax [ yamley dy 
0 0 


oO poo a 
= 4 i x2n by 2m ol") dy dy. 
0 /0 
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Changing variables to plane polar coordinates (p, @) given by x = pcos ¢, y = psing, we 
obtain 


n/2 po “s 
T(n)T(m) = 4 i printnV ee sin2”"—! d cos2"-! pdp do 
0 0 


n/2 ioe) ‘s 
= 4 sin”! o cos2"—! oad | printn—l ep" dp 
0 0 
= B(m,n)T (m+ n), 
where in the last line we have used the results (18.157) and (18.164). < 


The result (18.165) is useful in proving the identity (18.158) satisfied by the 
gamma function, since 


oO nl 
y" dy 
T(n)l(1 —n) = BI — n,n) = | : 
o ity 


where, in the second equality, we have used the integral representation (18.163). 


For 0 <n < 1 this integral can be evaluated using contour integration and has 
the value 2/(sinnz) (see exercise 24.19), thereby proving result (18.158) for this 
range of n. Extensions to other ranges require more sophisticated methods. 


18.12.3 The incomplete gamma function 


In the definition (18.153) of the gamma function, we may divide the range of 
integration into two parts and write 


x ‘00 
T(n) = | ue du+ / ule du = y(n, x) + T(n, x), 
7 x (18.166) 


whereby we have defined the incomplete gamma functions y(n,x) and I(n,x), 
respectively. The choice of which of these two functions to use is merely a matter 
of convenience. 


> Show that if n is a positive integer 


n Xe salt 
T(n, x) = (n—1)!e~* kt 


oa 
IL 


From (18.166), on integrating by parts we find 


ice) ioe) 
T(n, x) = i ut edu = x" 'e* + (n— nf u’e"'du 
x x 


= x"1e* + (n— 1) (n— 1,x), 
which is valid for arbitrary n. If n is an integer, however, we obtain 


D(n, x) = e*[x"1 4 (n= 1)x"™? + (n— 1)(n— 2)x" 3 +--+ + (n—1)4] 


n—l 


a3 x 
=(n-Ile*>> a 
k=0 
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which is the required result. < 


We note that is it conventional to define, in addition, the functions 


T(a,x) 
T(a) ’ 


y(a, x) 


P(a,x)= T@)’ 


Q(a, x) = 


which are also often called incomplete gamma functions; it is clear that Q(a, x) = 
1 — P(a,x). 


18.12.4 The error function 


Finally, we mention the error function, which is encountered in probability theory 
and in the solutions of some partial differential equations. The error function is 
related to the incomplete gamma function by erf(x) = 15x) Jt and is thus 
given by 


2 ey 2 2 a 2 
erf(x) = = e“ du=1— zal e du. (18.167) 


From this definition we can easily see that 
erf(0) = 0, erf(oo) = 1, erf(—x) = —erf(x). 


By making the substitution y = ./2u in (18.167), we find 


ox 
erf(x) = i2{ evi? dy. 
T JO 


The cumulative probability function ®(x) for the standard Gaussian distribution 
(discussed in section 30.9.1) may be written in terms of the error function as 
follows: 


It is also sometimes useful to define the complementary error function 


(5,2) 


Va 


a fs 
erfe(x) = 1 — erf(x) = =| e du= (18.168) 
VE Js 


18.13 Exercises 


18.1 Use the explicit expressions 
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o_ /4 o /3 
Yo = 4/ a> Yy = / 7 0088, 


y#! = F,/ = sin 0 exp(+i¢), v9 = ,/£ (cos? 0-1), 


ye = F/# sin 0 cos @ exp(+id), ye? = / > sin’ 0 exp(+2id), 


to verify for 7 = 0,1,2 that 


my a 
a7 (0,.9)|" 


m=—¢ 


and so is independent of the values of 0 and @. This is true for any /, but 
a general proof is more involved. This result helps to reconcile intuition with 
the apparently arbitrary choice of polar axis in a general quantum mechanical 
system. 
18.2 Express the function 
f (0, b) = sin O[sin?(0/2) cos + icos?(8/2) sin o] + sin?(@/2) 
as a sum of spherical harmonics. 
18.3 Use the generating function for the Legendre polynomials P,,(x) to show that 
1 
= “ (2n)! 
‘p Ponyi(x)dx = (—1) =Inkn+ 1! 


and that, except for the case n = 0, 


1 
| P(x) dx = 0. 
0 


18.4 Carry through the following procedure as a proof of the result 


2 
2n+1° 


1 
In =a P,(z)Pn(z) dz = 
-l 


(a) Square both sides of the generating-function definition of the Legendre 
polynomials, 


(1—2zht+ hy? = => P,2) ya". 


n=0 


(b) Express the RHS as a sum of powers of h, obtaining expressions for the 
coefficients. 

(c) Integrate the RHS from —1 to 1 and use the orthogonality property of the 
Legendre polynomials. 

(d) Similarly integrate the LHS and expand the result in powers of h. 

(e) Compare coefficients. 


18.5 The Hermite polynomials H,,(x) may be defined by 


(x, h) = exp(2xh — h?) Yo Hac 


n=0 
Show that 
eo o® o® 
of” 2x 4 9 =0 
Be oR on 
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18.6 


18.8 


and hence that the H,,(x) satisfy the Hermite equation 
y" — 2xy' + 2ny = 0, 


where n is an integer > 0. 
Use ® to prove that 


(a) H)(x) = 2nHy-1(x), 

(b) Hyst(x) — 2xHy(x) + 2nHy_1(x) = 0. 

A charge +2g is situated at the origin and charges of —q are situated at distances 
ta from it along the polar axis. By relating it to the generating function for the 


Legendre polynomials, show that the electrostatic potential ® at a point (r, 0, d) 
with r >a is given by 


Or, 0,) = i Lo y° P,,(cos 0). 


For the associated Laguerre polynomials, carry through the following exercises. 


(a) Prove the Rodrigues’ formula 

ex ™ d" 
n! dx" 

taking the polynomials to be defined by 


(n aa k 
Li(x pS Aes Sine, , 
>a Ge ktm! 


Li(x) = 


( mm os) 


> 


(b) Prove the recurrence relations 
(n+ 1)LM (x) = (2n-+.m + 1 — x)LM(x) — (n+ mL (x), 
x(L")'(x) = nLbin(x) — (n+ m)L™_, (x), 
but this time taking the polynomial as defined by 


d™ 
L(x) = (—1)" Luan) 


or the generating function. 


The quantum mechanical wavefunction for a one-dimensional simple harmonic 
oscillator in its nth energy level is of the form 


w(x) = exp(—x?/2)H (x), 


where H,(x) is the nth Hermite polynomial. The generating function for the 
polynomials is 


2H; 
Gah =e ye as dn 
n=0 


(a) Find H;(x) for i = 1,2,3,4. 
(b) Evaluate by direct calculation 


i “ee H,(x)H,(x) dx, 


(i) for p = 2, q = 3; (ii) for p = 2, q = 4; (iii) for p = q = 3. Check your 
answers against the expected values 2? pljn Ong: 


642 


18.13 EXERCISES 


18.9 


18.10 


18.11 


18.12 


18.13 


18.14 


[ You will find it convenient to use 


2 neo” dx = (2n)! /t 


22nn! 


for integer n > 0.] 
By initially writing y(x) as x!/?f(x) and then making subsequent changes of 
variable, reduce Stokes’ equation, 
fy. 
he + Axy = 0, 


to Bessel’s equation. Hence show that a solution that is finite at x = 0 is a 


multiple of x!/7J1/3(3.V2x?). 
By choosing a suitable form for h in their generating function, 


G(z,h) = exp E (1 i) | = > J,(z)h", 


show that integral repesentations of the Bessel functions of the first kind are 
given, for integral m, by 


__1)m 2a 
Jon (Z) = (-) i cos(z cos 8) cos 2m0 dé m>1, 
1 0 
(-1)""! 2n 
Jom4i(Z) = - | cos(z cos @) sin(2m+1)0d@ m>0. 
0 


Identify the series for the following hypergeometric functions, writing them in 
terms of better known functions: 

(a) F(a,b,b;z), 

(b) F(1,1,2;—x), 

(c) F(5,135—x*), 
dy FG 35530) 


(e) F(—a,a, 35 sin? x); this is a much more difficult exercise. 


> 


3 
2 
3 
2 


By making the substitution z = (1 — x)/2 and suitable choices for a, b and c, 
convert the hypergeometric equation, 


2 
ET ne aa Pe eae ee oe —abu = 0, 
dz? dz 
into the Legendre equation, 
Py, dy 
1—x°)—5 — 2x +: 40 + ly =0. 
(1) — 2x $0 + Dy 


Hence, using the hypergeometric series, generate the Legendre polynomials P,(x) 
for the integer values 7 = 0,1,2,3. Comment on their normalisations. 
Find a change of variable that will allow the integral 


_ - u—1 
r= f Gri 


to be expressed in terms of the beta function, and so evaluate it. 
Prove that, if m and n are both greater than —1, then 


cae u™ Gis T4m+ HIT 44 1)] 
Jy (au2 + byertn+2)/2 8" Jaq lm+1)/2 pln+t)/2 Tia(m+n+2)]° 
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18.15 


18.16 


18.17 


18.18 


18.19 


Deduce the value of 


_ (oe) (u+2)y 
J= ) a du. 


The complex function z! is defined by 
z= [ wetdu for Rez > —1. 
0 


For Re z < —1 it is defined by 
(z +n)! 
(z+tn\(z+n—1)---(24+1)7 
where n is any (positive) integer > —Re z. Being the ratio of two polynomials, z! 


is analytic everywhere in the finite complex plane except at the poles that occur 
when z is a negative integer. 


(a) Show that the definition of z! for Re z < —1 is independent of the value of 
n chosen. 

(b) Prove that the residue of z! at the pole z = —m, where m is an integer > 0, 
is (—1)""'/(m—1)!. 


For —1 < Rez < 1, use the definition and value of the beta function to show 


that 
0 uz 
'(-z)!= — d 
z\(—z) [ (+o U 


Contour integration gives the value of the integral on the RHS of the above 
equation as mz cosec 2z. Use this to deduce the value of (—$)}. 
The integral 


pa eee 
=. ——— * 
i... , (8) 


in which a > 0, occurs in some statistical mechanics problems. By first considering 


the integral 
J= i eiulk-+ia) dy 
0 


and a suitable variation of it, show that J = (1/a) exp(a’)erfc(a), where erfc(x) 
is the complementary error function. 
Consider two series expansions of the error function as follows. 


(a) Obtain a series expansion of the error function erf(x) in ascending powers 
of x. How many terms are needed to give a value correct to four significant 
figures for erf(1)? 

(b) Obtain an asymptotic expansion that can be used to estimate erfc(x) for 
large x (> 0) in the form of a series 

00 
erfc(x) = R(x) = eo an 
x" 

n=0 

Consider what bounds can be put on the estimate and at what point the 

infinite series should be terminated in a practical estimate. In particular, 

estimate erfc(1) and test the answer for compatibility with that in part (a). 


For the functions M(a,c;z) that are the solutions of the confluent hypergeometric 
equation, 
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18.20 


18.21 


18.22 


18.23 


(a) use their series representation to prove that 
d 
b Gz Mlaesz) =aM(a+1,c+1;z); 


(b) use an integral representation to prove that 
M(a,c;z) =e’ M(c — a,c;—z). 


The Bessel function J,(z) can be considered as a special case of the solution 
M(a,c;z) of the confluent hypergeometric equation, the connection being 


lim M(a,v + 1;~2/a) = 2"? J,(2,/2). 


a0 T(v +1) 
Prove this equality by writing each side in terms of an infinite series and showing 
that the series are the same. 
Find the differential equation satisfied by the function y(x) defined by 


y(x) = Ax | et"! dt = Ax"y(n, x), 
0 


and, by comparing it with the confluent hypergeometric function, express y as 
a multiple of the solution M(a,c;z) of that equation. Determine the value of A 
that makes y equal to M. 

Show, from its definition, that the Bessel function of the second kind, and of 
integral order v, can be written as 


wtey= 2 [Med (aya) 
H=Vv 


1 


Using the explicit series expression for J,(z), show that 0J,(z)/0u can be written 
as 


Jo(z) in (5) + g(v,2), 


and deduce that Y,(z) can be expressed as 


2 Zz 
¥,(2) = <J,(2)In (5) +h(v,2), 
where h(v,z), like g(v,z), is a power series in z. 


Prove two of the properties of the incomplete gamma function P(a, x”) as follows. 


(a) By considering its form for a suitable value of a, show that the error function 
can be expressed as a particular case of the incomplete gamma function. 

(b) The Fresnel integrals, of importance in the study of the diffraction of light, 
are given by 


C(x) = [ ($2) dt, S(x) = if sin (57?) dt. 


Show that they can be expressed in terms of the error function by 


C(x) + iS(x) = A erf Fa =| : 


where A is a (complex) constant, which you should determine. Hence express 
C(x) + iS(x) in terms of the incomplete gamma function. 


645 


SPECIAL FUNCTIONS 


18.24 


18.1 
18.3 


18.9 


18.11 


18.13 


The solutions y(x,a) of the equation 


are known as parabolic cylinder functions. 


(a) If y(x,a) is a peer of (*), determine which of the following are also 
solutions: (i) y(a,—x), (ii) y(—a, x), (iti) y(a,ix) and (iv) y(—a, ix). 
(b) Show that one solution of (*), even in x, is 


=) 
vi(xa) =e 4M(ha + 4,4, 4%), 


where M(a,c,z) is the confluent hypergeometric function satisfying 
0M 
a 
You may assume (or prove) that a second solution, odd in x, is given by 
= xo 2/4 3.3 142 
yo(x,a) = xe "4M (4 54 + 5,55 5X"): 
(c) Find, as an infinite series, an explicit expression for e* /*y;(x, a). 
(d) Using the results from part (a), show that y;(x,a) can also be written as 


+(c — 2) om = 0. 


2/4 


yilx,a) =e" /4*M(-4 54+ 3,552). 
(e) By making a suitable choice for a deduce that 


= by, (— 1) by 
1+ Om a ~o (145° a ‘), 


n=1 


where b, = [J/_,(2r — 3). 


18.14 Hints and answers 


Note that taking the square of the modulus eliminates all mention of ¢. 
Integrate both sides of the generating function definition from x = 0 to x = 1, 
and then expand the resulting ae (1+h?)'/?, using a binomial expansion. Show 
that '/7C,, can be written as [(—1)"—!(2m — 2)!]/[2?"-'m!(m — 1)!]. 

Prove the stated equation using fe explicit closed form of the generating function. 
Then substitute the series and require the coefficient of each power of h to vanish. 
(b) Differentiate result (a) and then use (a) again to replace the derivatives. 

(a) Write the result of using Leibnitz’ theorem on the product of x"*" and e~™ asa 
finite sum, evaluate the separated derivatives, and then re-index the summation. 
(b) For the first recurrence relation, differentiate the generating function with 
respect to h and then use the generating function again to replace the exponential. 
Equating coefficients of h” then yields the result. For the second, differentiate the 
corresponding relationship for the ordinary Laguerre polynomials m times. 

xf" + xf! + (Ax? — 4)f =0. Then, in turn, set x3? =u, and 34'u = v; then v 
satisfies Bessel’s equation with v = i. 

(a) 1— ‘am a (b) x A ts +x). (c) Compare the calculated coefficients with those 
of tan—! x. F(z, .35-x?) = xf tan! x. (d) x71 sin“! x. (e) Note that a term 
containing x?" can only arise from the first, n+ 1 terms of an expansion in powers 
of sin? x; make a few trials. F(—a, a, 5 z ; sin 2x) = = cos 2ax. 

Looking for f(x) =u such that u+ 1 is an inverse power of x with f(0) = oo and 


f(1) = 1 leads to f(x) = 2x7! — 1. 1 = B($, 3)/V2 = 2/(2 V2). 
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18.15 


18.17 


18.19 


18.21 


18.23 


(a) Show that the ratio of two definitions based on m and n, with m > n > —Re z, 
is unity, independent of the actual values of m and n. 
(b) Consider the limit as z ~ —m of (z+ m)z!, with the definition of z! based on 
n where n > m. 
Express the integrand in partial fractions and use J, as given, and J’ = 
to exp[—iu(k —ia)] du to express I as the sum of two double integral expressions. 
Reduce them using the standard Gaussian integral, and then make a change of 
variable 2v = u + 2a. 
(b) Using the representation 
1 
M(a,b;2) =) _ | et! (1 — 1)?! dt 
T'(b—a)T(a) Jo 
allows the equality to be established, without actual integration, by changing the 
integration variable to s = 1 —t. 
Calculate y’(x) and y"(x) and then eliminate x~'e~* to obtain xy” +(n+14+x)y’+ 
ny = 0; M(n,n+ 1;—x). Comparing the expansion of the hypergeometric series 
with the result of term by term integration of the expansion of the integrand 
shows that A =n. 
(a) If the dummy variable in the incomplete gamma function is t, make the change 
of variable y = + /t. Now choose a so that 2(a — 1) + 1 = 0; erf(x) P(5,x°). 
(b) Change the integration variable u in the standard representation of the RHS 
to s, given by u= 5/n(1 — i)s, and note that (1 — i)? = —2i. A = (14+ i)/2. From 
part (a), C(x) + iS(x) = $(1+)P(4,—47i x’). 
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Quantum operators 


Although the previous chapter was principally concerned with the use of linear 
operators and their eigenfunctions in connection with the solution of given 
differential equations, it is of interest to study the properties of the operators 
themselves and determine which of them follow purely from the nature of the 
operators, without reference to specific forms of eigenfunctions. 


19.1 Operator formalism 


The results we will obtain in this chapter have most of their applications in the 
field of quantum mechanics and our descriptions of the methods will reflect this. 
In particular, when we discuss a function y that depends upon variables such as 
space coordinates and time, and possibly also on some non-classical variables, y 
will usually be a quantum-mechanical wavefunction that is being used to describe 
the state of a physical system. For example, the value of |p|? for a particular 
set of values of the variables is interpreted in quantum mechanics as being the 
probability that the system’s variables have that set of values. 

To this end, we will be no more specific about the functions involved than 
attaching just enough labels to them that a particular function, or a particular 
set of functions, is identified. A convenient notation for this kind of approach 
is that already hinted at, but not specifically stated, in subsection 17.1, where 
the definition of an inner product is given. This notation, often called the Dirac 
notation, denotes a state whose wavefunction is y by |p); since y belongs to a 
vector space of functions, | y) is known as a ket vector. Ket vectors, or simply kets, 
must not be thought of as completely analogous to physical vectors. Quantum 
mechanics associates the same physical state with ke'|y) as it does with |p) 
for all real k and 0 and so there is no loss of generality in taking k as 1 and 0 
as 0. On the other hand, the combination c| y;) + c2| y2), where | y,) and | p>) 


648 


19.1 OPERATOR FORMALISM 


represent different states, is a ket that represents a continuum of different states 
as the complex numbers c; and c» are varied. 

If we need to specify a state more closely — say we know that it corresponds 
to a plane wave with a wave number whose magnitude is k — then we indicate 
this with a label; the corresponding ket vector would be written as | k). If we also 
knew the direction of the wave then |k) would be the appropriate form. Clearly, 
in general, the more labels we include, the more precisely the corresponding state 
is specified. 

The Dirac notation for the Hermitian conjugate (dual vector) of the ket vector 
|) is written as (y| and is known as a bra vector; the wavefunction describing this 
state is py", the complex conjugate of y. The inner product of two wavefunctions 
{ w* ¢ dv is then denoted by (y| d) or, more generally if a non-unit weight function 
p is involved, by 


(p| p| ’), evaluated as [vom dr. (19.1) 


Given the (contrived) names for the two sorts of vectors, an inner product like 
(p| @) becomes a particular type of ‘bra(c)ket’. Despite its somewhat whimsical 
construction, this type of quantity has a fundamental role to play in the interpre- 
tation of quantum theory, because expectation values, probabilities and transition 
rates are all expressed in terms of them. For physical states the inner product of 
the corresponding ket with itself, with or without an explicit weight function, is 
non-zero, and it is usual to take 


(ply) = 1. 


Although multiplying a ket vector by a constant does not change the state 
described by the vector, acting upon it with a more general linear operator A 
results (in general) in a ket describing a different state. For example, if p is a state 
that is described in one-dimensional x-space by the wavefunction (x) = exp(—x?) 
and A is the differential operator 0/0x, then 


lyi) = Alp) = | Ay) 


is the ket associated with the state whose wavefunction is y1(x) = —2x exp(—x?), 
clearly a different state. This allows us to attach a meaning to an expression such 
as (|A| p) through the equation 


(P/Al p) = (Pl yr), (19.2) 


i.e. it is the inner product of | y,) and | ¢). We have already used this notation in 
equation (19.1), but there the effect of the operator A was merely multiplication 
by a weight function. 

If it should happen that the effect of an operator acting upon a particular ket 
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is to produce a scalar multiple of that ket, ie. 
Aly) =Aly), (19.3) 


then, just as for matrices and differential equations, | y) is called an eigenket or, 
more usually, an eigenstate of A, with corresponding eigenvalue 4; to mark this 
special property the state will normally be denoted by |), rather than by the 
more general | y). Taking the Hermitian conjugate of this ket vector eigenequation 
gives a bra vector equation, 


(plAT = A* (yl. (19.4) 


It should be noted that the complex conjugate of the eigenvalue appears in this 
equation. Should the action of A on |y) produce an unphysical state (usually 
one whose wavefunction is identically zero, and is therefore unacceptable as a 
quantum-mechanical wavefunction because of the required probability interpre- 
tation) we denote the result either by 0 or by the ket vector |@) according to 
context. Formally, |) can be considered as an eigenket of any operator, but one 
for which the eigenvalue is always zero. 

If an operator A is Hermitian (At = A) then its eigenvalues are real and 
the eigenstates can be chosen to be orthogonal; this can be shown in the same 
way as in chapter 17 (but using a different notation). As indicated there, the 
reality of their eigenvalues is one reason why Hermitian operators form the 
basis of measurement in quantum mechanics; in that formulation of physics, the 
eigenvalues of an operator are the only possible values that can be obtained when 
a measurement of the physical quantity corresponding to the operator is made. 
Actual individual measurements must always result in real values, even if they 
are combined in a complex form (x + iy or re”) for final presentation or analysis, 
and using only Hermitian operators ensures this. The proof of the reality of the 
eigenvalues using the Dirac notation is given below in a worked example. 

In the same notation the Hermitian property of an operator A is represented 
by the double equality 


(A dlp) = (PlAl p) = (AA y). 


It should be remembered that the definition of an Hermitian operator involves 
specifying boundary conditions that the wavefunctions considered must satisfy. 
Typically, they are that the wavefunctions vanish for large values of the spatial 
variables upon which they depend; this deals with most physical systems since 
they are nearly all formally infinite in extent. Some model systems require the 
wavefunction to be periodic or to vanish at finite values of a spatial variable. 
Depending on the nature of the physical system, the eigenvalues of a particular 
linear operator may be discrete, part of a continuum, or a mixture of both. For 
example, the energy levels of the bound proton-electron system (the hydrogen 
atom) are discrete, but if the atom is ionised and the electron is free, the energy 
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spectrum of the system is continuous. This system has discrete negative and 
continuous positive eigenvalues for the operator corresponding to the total energy 
(the Hamiltonian). 


» Using the Dirac notation, show that the eigenvalues of an Hermitian operator are real. 


Let |a) be an eigenstate of Hermitian operator A corresponding to eigenvalue a, then 


Ala) = ala), 

= (alAla) = (ala|a) = a(ala), 
and 

(alAt = a*(al, 

=> (alAtla) = a’(ala), 
(a|A|a) = a*(al\a), since A is Hermitian. 
Hence, 

(a—a'){a\a) = 0, 


=> a 


a’, since (a\a) #0. 


Thus a is real. < 


It is not our intention to describe the complete axiomatic basis of quantum 
mechanics, but rather to show what can be learned about linear operators, and 
in particular about their eigenvalues, without recourse to explicit wavefunctions 
on which the operators act. 

Before we proceed to do that, we close this subsection with a number of results, 
expressed in Dirac notation, that the reader should verify by inspection or by 
following the lines of argument sketched in the statements. Where a sum over 
a complete set of eigenvalues is shown, it should be replaced by an integral for 
those parts of the eigenvalue spectrum that are continuous. With the notation 
that |a,) is an eigenstate of Hermitian operator A with non-degenerate eigenvalue 
dy (or, if a, is k-fold degenerate, then a set of k mutually orthogonal eigenstates 
has been constructed and the states relabelled), we have the following results. 


Al Gn) = ay| an), 
(am|Gn) =6Omn  (orthonormality of eigenstates), (19.5) 


A(Cn| Gn) + Cl Qm)) = Cndnl Gn) + Cm4ml| Gm) — (linearity). (19.6) 


The definitions of the sum and product of two operators are 


(A+ B)|p) = Alp) + Bly), (19.7) 
AB\|y) = A(B|p)) (# BA|y) in general), (19.8) 
= A?la,) =a?| ay). (19.9) 
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If A| a,) = ajay) for all Ny <n < No, then 


Np 
|p) = s d,| dn) satisfies A| yp) = aly) for any set of dj. 
n=N, 


For a general state | y), 


a 


|v) = So eal an), where cy = (dnl). (19.10) 
n=0 


This can also be expressed as the operator identity, 


1= 5a (ag), (19.11) 


in the sense that 


lw) =1]y) = sa (anly) a5 ce 


n=0 n=0 
It also follows that 


1= (plp) = (>: Cn at) (>: ola) = Se citron = a lenl?. 


m=0 n=0 m,n n=0 (19.12) 


Similarly, the expectation value of the physical variable corresponding to A is 


oe) ioe) 


(plAl py = S> eh, (aml Al anon = S— (nl Onl Gn) Cn 


mn myn 


& 
= be mCndnOmn = “> lenl? an. (19.13) 


mn 


19.1.1 Commutation and commutators 


As has been noted above, the product AB of two linear operators may or may 
not be equal to the product BA. That is 


AB|w) is not necessarily equal to BA|). 


If A and B are both purely multiplicative operators, multiplication by f(r) 
and g(r) say, then clearly the order of the operations is immaterial, the result 
| f(r)g(r)p) being obtained in both cases. However, consider a case in which A 
is the differential operator 0/0x and B is the operator ‘multiply by x’. Then the 
wavefunction describing AB| y) is 


0 Ow 
s~ (x(x) = v(x) +x, 
Ox Ox 
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whilst that for BA| y) is simply 


which is not the same. 
If the result 


AB\|p) = BA|p) 


is true for all ket vectors |), then A and B are said to commute; otherwise they 
are non-commuting operators. 

A convenient way to express the commutation properties of two linear operators 
is to define their commutator, [A,B], by 


[4,B]|p) = AB|p) — BAly). (19.14) 


Clearly two operators that commute have a zero commutator. But, for the example 
given above we have that 


| 2.x] vo = (woo + x22) — (x2) = pay = 1 x y 
Ox OX x 


or, more simply, that 


0 
|| =F (19.15) 
Ox 


in words, the commutator of the differential operator 0/0x and the multiplicative 
operator x is the multiplicative operator 1. It should be noted that the order of 
the linear operators is important and that 


[A,B] =—[B,A]. (19.16) 


Clearly any linear operator commutes with itself and some other obvious zero 
commutators (when operating on wavefunctions with ‘reasonable’ properties) are: 


[A,I], where I is the identity operator; 
[A",A”], for any positive integers n and m; 
[ A, p(A)], where p(x) is any polynomial in x; 
[A,c], where A is any linear operator and c is any constant; 
[ f(x), g(x)], where the functions are mutiplicative; 


[ A(x), B(y)], where the operators act on different variables, with 


poe as a specific example 
Ox’ dy E ps 
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Simple identities amongst commutators include the following: 


[4,B+C]=[4,B]+[4,C], (19.17) 
[4+B,C]=[4,C]+[B,C], (19.18) 
[A, BC] = ABC — BCA + BAC — BAC 
= (AB — BA)C + B(AC — CA) 
=[4,B]C+B[A4,C], (19.19) 
[AB,C]=A[B,C]+[4,C]B. (19.20) 


>If A and B are two linear operators that both commute with their commutator, prove that 
[A, B"] = nB""[ A,B] and that [A", B] = nA"! [A, B]. 


Define C, by C, = [A, B"]. We aim to find a reduction formula for C,,: 


C, = [A,B B""] 
= [4,B] B""'! + B[ A,B" ], using (19.19), 
= B"'[A,B]+B[4,B""], since [[4,B],B] =0, 
= B""'[A,B]+ BC,_1, the required reduction formula, 
= B"'[ A,B] + B{B"* [A,B] +BC,_}, applying the formula, 
= 2B"! [A,B] + BC,» 


= nB""'[A,B]+ B"Cp. 


However, Cy = [A,I ] = 0 and so C, = nB""![ A,B]. 
Using equation (19.16) and interchanging A and B in the result just obtained, we find 


[4",B] =—[B,4"] = na"! [B,A] =n4""[4,B], 


as stated in the question. < 


As the power of a linear operator can be defined, so can its exponential; 
this situation parallels that for matrices, which are of course a particular set of 
operators that act upon state functions represented by vectors. The definition 
follows that for the exponential of a scalar or matrix, namely 


2 n 
expA => _ (19.21) 
n=0 . 


Related functions of A, such as sin A and cos A, can be defined in a similar way. 

Since any linear operator commutes with itself, when two functions of it 
are combined in some way, the result takes a form similar to that for the 
corresponding functions of scalar quantities. Consider, for example, the function 
f(A) defined by f(A) = 2 sin Acos A. Expressing sin A and cos A in terms of their 
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defining series, we have 


(ee) 


S (— 1) m 42m+1 (—1)" Ann 
f(A) = 2 eT (Qm+1! —~ Qn)! 


Writing m+n as r and replacing n by s, we have 
1) r—s a 
A) =? Atti (— 
F(A) s (>: (2r —2s+ 1)! (2s)! 
= 2y7(-1) Cp Att 
r=0 


where 


yy 


A ae 1 = 2r+1 
a “Gait Del ~ Gai ry Cas. 


= 
By adding the binomial expansions of 27+! = (1 + 1)?"+! and 0 = (1 — 1)?'*4, it 
can easily be shown that 
grt = 22 a+, —< C= 
s=0 


It then follows that 


—1) Aartl 92r -> (— 1)" (2A)?! 


Qr+1)! Gren ee 


2sin AcosA = 25° ( 
r=0 


a not unexpected result. 

However, if two (or more) linear operators that do not commute are involved, 
combining functions of them is more complicated and the results less intuitively 
obvious. We take as a particular case the product of two exponential functions 
and, even then, take the simplified case in which each linear operator commutes 
with their commutator (so that we may use the results from the previous worked 
example). 


>If A and B are two linear operators that both commute with their commutator, show that 


exp(A) exp(B) = exp(A + B+ 4[A,B]). 


We first find the commutator of A and exp AB, where J is a scalar quantity introduced for 
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later algebraic convenience: 


[4,e"?] = 


4,S~ ae =4 [ A, B"] 


n=0 


roe) 


a : ‘ 
> a nB"' [A,B], using the earlier result, 


2 yn 
= nB" [A,B] 


oa) 


pm Bm dk 
= 15 — [A, B], writing m=n—1, 


m=0 
=)e*® [A,B]. 
Now consider the derivative with respect to / of the function 
f(A) = C4 0B op HAtB). 


In the following calculation we use the fact that the derivative of e*© is Ce’; this is the 
same as e*C, since any two functions of the same operator commute. Differentiating the 
three-factor product gives 


af = eA fe'B eHUAFB) 4 gi iB Be AA+B) 4 giAg/B(_4 — B)e*4+8) 
di 
= e4(e8 A + Je" [A,B] e748) 4 04 eB Be-HA+B) 
— pA GAB A g-HAtB) __ pA G/B Bo—MA+B) 

_ 4) eB [A,B] oe MATB) 
= A[A,B] f(A). 
In the second line we have used the result obtained above to replace Ae’, and in the last 
line have used the fact that [A,B] commutes with each of A and B, and hence with any 
function of them. 

Integrating this scalar differential equation with respect to 4 and noting that f(0) = 1, 
we obtain 


Inf = 1p [A,B] => 7A g/B e—HATB) — fa= et FP lAB). 
Finally, post-multiplying both sides of the equation by e“4+¥) and setting 2 = 1 yields 


1 
eAe8 = eiIABHATB 


19.2 Physical examples of operators 


We now turn to considering some of the specific linear operators that play a 
part in the description of physical systems. In particular, we will examine the 
properties of some of those that appear in the quantum-mechanical description 
of the physical world. 

As stated earlier, the operators corresponding to physical observables are re- 
stricted to Hermitian operators (which have real eigenvalues) as this ensures the 
reality of predicted values for experimentally measured quantities. The two basic 
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quantum-mechanical operators are those corresponding to position r and mo- 
mentum p. One prescription for making the transition from classical to quantum 
mechanics is to express classical quantities in terms of these two variables in 
Cartesian coordinates and then make the component by component substitutions 


r— multiplicative operatorr and p-— differential operator — ihV. 
(19.22) 


This generates the quantum operators corresponding to the classical quantities. 
For the sake of completeness, we should add that if the classical quantity contains 
a product of factors whose corresponding operators A and B do not commute, 
then the operator 5(AB + BA) is to be substituted for the product. 

The substitutions (19.22) invoke operators that are closely connected with the 
two that we considered at the start of the previous subsection, namely x and 
0/0x. One, x, corresponds exactly to the x-component of the prescribed quantum 
position operator; the other, however, has been multiplied by the imaginary 
constant —ih, where fi is the Planck constant divided by 2z. This has the (subtle) 
effect of converting the differential operator into the x-component of an Hermitian 
operator; this is easily verified using integration by parts to show that it satisfies 
equation (17.16). Without the extra imaginary factor (which changes sign under 
complex conjugation) the two sides of the equation differ by a minus sign. 

Making the differential operator Hermitian does not change in any essential 
way its commutation properties, and the commutation relation of the two basic 
quantum operators reads 


0 
[px.x] = [=n = —ih. (19.23) 
Ox 


Corresponding results hold when x is replaced, in both operators, by y or z. 
However, it should be noted that if different Cartesian coordinates appear in the 
two operators then the operators commute, i.e. 


(p01 = [paz] =p) = (paz |) = lPaxl= lee l= 0. 
(19.24) 


As an illustration of the substitution rules, we now construct the Hamiltonian 
(the quantum-mechanical energy operator) H for a particle of mass m moving 
in a potential V(x,y,z) when it has one of its allowed energy values, ie its 
energy is E,, where H|y,) = E,|yn). This latter equation when expressed in a 
particular coordinate system is the Schrédinger equation for the particle. In terms 
of position and momentum, the total classical energy of the particle is given by 
p _ Pe +P, +Pz 


mm + V(x, y, 2) = a) on + V(x, y, 2). 


Substituting —ihd/0x for p, (and similarly for p, and p-) in the first term on the 


E= 
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RHS gives 
(—ih? 6 2 | (-ih? 0 6 |, (ih? @ A 
2m 0x 0x 2m Oyéy ' 2m azéz 


The potential energy V, being a function of position only, becomes a purely 
multiplicative operator, thus creating the full expression for the Hamiltonian, 


hn? ( e re a 
A= t V(x, y,Z), 
2m (= bs oy? Bs 2) (952) 


and giving the corresponding Schrédinger equation as 


i? On Own Wn 
Ayn = am ( axe t ay? t Oz ) t V(X, y,Z)Wn = Enn. 


We are not so much concerned in this section with solving such differential 
equations, but with the commutation properties of the operators from which they 
are constructed. To this end, we now turn our attention to the topic of angular 
momentum, the operators for which can be constructed in a straightforward 
manner from the two basic sets. 


19.2.1 Angular momentum operators 


As required by the substitution rules, we start by expressing angular momentum 
in terms of the classical quantities r and p, namely L = r x p with Cartesian 
components 


L;=Xpy—ypx, Lx =ypz—Zpy, Ly = zpx — Xpz. 


Making the substitutions (19.22) yields as the corresponding quantum-mechanical 
operators 


: 
teh ( if = =) ; (19.25) 


It should be noted that for xp,, say, x and 0/dy commute, and there is no 
ambiguity about the way it is to be carried into its quantum form. Further, since 
the operators corresponding to each of its factors commute and are Hermitian, 
the operator corresponding to the product is Hermitian. This was shown directly 
for matrices in exercise 8.7, and can be verified using equation (17.16). 

The first question that arises is whether or not these three operators commute. 
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Consider first 


=—h? ae : ne —7? a + 2x a 
= Vax azax 322 7 Bydx  *Syaz ) 


a a a 0 a? 
Rr 2 | + 
=— zy Zz xyrset+x—+xz : 
Ox0z Oxdy ay oy 0zoy 


These two expressions are not the same. The difference between them, ie. the 
commutator of L, and Ly, is given by 


a 
oy 


6) 
| Ly Ly | = bedly —- Ly Lge (« rx) = ihL,. (19.26) 


This, and two similar results obtained by permutting x, y and z cyclically, 
summarise the commutation relationships between the quantum operators corre- 
sponding to the three Cartesian components of angular momentum: 


= ihL., (19.27) 


As well as its separate components of angular momentum, the total angular 
momentum associated with a particular state |) is a physical quantity of interest. 
This is measured by the operator corresponding to the sum of squares of its 
components, 


LD? = Le +L) +L. (19.28) 


This is an Hermitian operator, as each term in it is the product of two Hermitian 
operators that (trivially) commute. It might seem natural to want to ‘take the 
square root’ of this operator, but such a process is undefined and we will not 
pursue the matter. 

We next show that, although no two of its components commute, the total 
angular momentum operator does commute with each of its components. In the 
proof we use some of the properties (19.17) to (19.20) and result (19.27). We begin 
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with 


[E.b.| =[Li+15 +15 6, | 
= Ly [Ly, Lz] + [Lx, Lz] Lx 
+ Ly [LyLe] + [Ly Lz] Ly + [13.12] 
= L,(—ih)Ly + (—ih) Ly Ly + Ly(ih)Ly + (iA)LyLy +0 
=0. 


Thus operators L? and L, commute and, continuing in the same way, it can be 
shown that 


[ibe S| ba | = 0 (19.29) 


Eigenvalues of the angular momentum operators 
We will now use the commutation relations for L? and its components to find 
the eigenvalues of L? and L-, without reference to any specific wavefunction. In 
other words, the eigenvalues of the operators follow from the structure of their 
commutators. There is nothing particular about L-, and Lx, or Ly could equally 
well have been chosen, though, in general, it is not possible to find states that are 
simultaneously eigenstates of two or more of Lx, Ly and L;. 
To help with the calculation, it is convenient to define the two operators 


C=, 411, and D= LiL, 


These operators are not Hermitian; they are in fact Hermitian conjugates, in that 
Ut = D and D* = U, but they do not represent measurable physical quantities. 
We first note their multiplication and commutation properties: 


UD = (Ly + iLy)(Ly — iLy) = LZ + Ly +i [ Ly, Lx] 
=L—-1?+AL,, (19.30) 

DU = (Ly — iLy)(Lx + iLy) = L} + Ly —i[ Ly, Ly] 
=L?—1?—-AL,, (19.31) 
[L.,U] =[Lz, Ly] +i[ Lz, Ly] =ihLy + AL, = AU, (19.32) 
[L.,D] =[L,,L,]—i[ L,,Ly] = ihLy — AL, = —AD. (19.33) 


In the same way as was shown for matrices, it can be demonstrated that if two 
operators commute they have a common set of eigenstates. Since L* and L, 
commute they possess such a set; let one of the set be |y) with 


L’|p) =alp) and L.|y) = bly). 
Now consider the state |y’) = U|y) and the actions of L? and L, upon it. 
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Consider first L?|1p’), recalling that L? commutes with both Ly and Ly and hence 
with U: 


Llp’) = L’U|p) = UL? |p) = Ualy) = aU|y) = aly’). 


Thus, |1p’) is also an eigenstate of L?, corresponding to the same eigenvalue as 
|). Now consider the action of L,: 


Lz|p') = L;U|y) 
= (UL, + AU)|yp), using [L,,U] = AU, 
= Ubly) + hUly) 
= (b+ h)U|p) 
=(b+h)\y’). 


Thus, |p’) is also an eigenstate of L,, but with eigenvalue b + h. 

In summary, the effect of U acting upon |p) is to produce a new state that 
has the same eigenvalue for L” and is still an eigenstate of L,, though with that 
eigenvalue increased by h. An exactly analogous calculation shows that the effect 
of D acting upon |y) is to produce another new state, one that also has the same 
eigenvalue for L? and is also still an eigenstate of L,, though with the eigenvalue 
decreased by fi in this case. For these reasons, U and D are usually known as 
ladder operators. 

It is clear that, by starting from any arbitrary eigenstate and repeatedly applying 
either U or D, we could generate a series of eigenstates, all of which have the 
eigenvalue a for L?, but increment in their L, eigenvalues by +h. However, we 
also have the physical requirement that, for real values of the z-component, its 
square cannot exceed the square of the total angular momentum, ie. b* < a. Thus 
b has a maximum value c that satisfies 


ec<a but (c+h) >a; 


let the corresponding eigenstate be |y,) with L,|y,) = clw,). Now it is still true 
that 
L,U|y) = (¢ +h)U|pu), 
and, to make this compatible with the physical constraint, we must have that 
U|yu) is the zero ket vector |@). Now, using result (19.31), we have 
DU|pu) = (L? — L? — hL:)|pu), 
=> 0/0) =D|0) =(@ —c —he)ly,), 
=> a=Cc(c+h). 


This gives the relationship between a and c. We now establish the possible forms 
for c. 
If we start with eigenstate |p,,), which has the highest eigenvalue c for L,, and 
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operate repeatedly on it with the (down) ladder operator D, we will generate a 
state |ya) which, whilst still an eigenstate of L? with eigenvalue a, has the lowest 
physically possible value, d say, for the eigenvalue of L,. If this happens after n 
operations we will have that d = c — nh and 


L,|pa) = (¢ — nh)|pa). 
Arguing in the same way as previously that D|y,) must be an unphysical ket 
vector, we conclude that 
0|0) = U|O) = UD\ypa) 
=(L?— 12 + AL,)|pa), using (19.30), 
= [a—(c— nh)? + h(e — nh) ]|a) 
a =(c—nh) —fi(c — nh). 


Equating the two results for a gives 
C+ch=C —2cnh+rh —ch+ nh’, 
2c(n + 1) = n(n + 1)h, 


ogee al! 
c = 5nh. 


Since n is necessarily integral, c is an integer multiple of sh. This result is valid 
irrespective of which eigenstate |y) we started with, though the actual value of 
the integer n depends on |y,,) and hence upon |). 

Denoting 5n by @ we can say that the possible eigenvalues of the operator 
L,, and hence the possible results of a measurement of the z-component of the 
angular momentum of a system, are given by 


th, (€—1)h, (€—2)h, ..., —Ch. 
The value of a for all 27 + 1 of the corresponding states, 


lu), Dip), D? lu), sees D™\wu), 


is ((¢ + 1)h?. 

The similarity of form between this eigenvalue and that appearing in Legendre’s 
equation is not an accident. It is intimately connected with the facts (i) that L? 
is a measure of the rotational kinetic energy of a particle in a system centred 
on the origin, and (ii) that in spherical polar coordinates L? has the same form 
as the angle-dependent part of V*, which, as we have seen, is itself proportional 
to the quantum-mechanical kinetic energy operator. Legendre’s equation and 
the associated Legendre equation arise naturally when V?p = f(r) is solved in 
spherical polar coordinates using the method of separation of variables discussed 
in chapter 21. 

The derivation of the eigenvalues /(/+1)h? and mh, with —¢ < m < 7, depends 
only on the commutation relationships between the corresponding operators. Any 
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other set of four operators with the same commutation structure would result 
in the same eigenvalue spectrum. In fact, quantum mechanically, orbital angular 
momentum is restricted to cases in which n is even and so / is an integer; this 
is in accord with the requirement placed on / if solutions to V7y = f(r) that are 
finite on the polar axis are to be obtained. The non-classical notion of internal 
angular momentum (spin) for a particle provides a set of operators that are able 
to take both integral and half-integral multiples of fh as their eigenvalues. 

We have already seen that, for a state |/,m) that has a z-component of 
angular momentum mh, the state U|/,m) is one with its z-component of angular 
momentum equal to (m+ 1)h. But the new state ket vector so produced is not 
necessarily normalised so as to make (/,m+1|¢7,m+ 1) = 1. We will conclude 
this discussion of angular momentum by calculating the coefficients 1, and v,, in 
the equations 


U|¢,m) = Umlf,m+1) and D\f,m) = v_\Z,m— 1) 


on the basis that (/,r|7,r) =1 for all 7 and r. 

To do so, we consider the inner product I = (¢,m|DU|7/,m), evaluated in two 
different ways. We have already noted that U and D are Hermitian conjugates 
and so I can be written as 


I a (@, m | UTU| t, m) = Ly (2m | t, m) Lm = [dnl 
But, using equation (19.31), it can also be expressed as 


I =(¢,m|L? — L? —AL,|¢,m) 
= (¢,m|¢(¢ + 1)h? — mh? — mh?| 2, m) 
= [C0 + 1h? — Wh? — mh? ] (¢,m| 2m) 
=[¢¢+1)—m(m4+1)]P. 


Thus we are required to have 


|Mml? = [4(¢ + 1) — m(m + 1) Pr, 


but can choose that all u,, are real and non-negative (recall that |m| < 7). A 
similar calculation can be used to calculate v,,. The results are summarised in the 
equations 


Ul¢,m) = (46+) —m(m Dhlé,m+ 1), (19.34) 


D\¢,m) = (77 +1) —m(m — Ih|¢,m— 1). (19.35) 
It can easily be checked that U|/,/) =|0) = D\¢,—/). 
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19.2.2 Uncertainty principles 


The next topic we explore is the quantitative consequences of a non-zero com- 
mutator for two quantum (Hermitian) operators that correspond to physical 
variables. 

As previously noted, the expectation value in a state |y) of the physical quantity 
A corresponding to the operator A is E[A] = (p| A|p). Any one measurement of 
A can only yield one of the eigenvalues of A. But if repeated measurements could 
be made on a large number of identical systems, a discrete or continuous range 
of values would be obtained. It is a natural extension of normal data analysis 
to measure the uncertainty in the value of A by the observed variance in the 
measured values of A, denoted by (AA)? and calculated as the average value of 
(A — E[A])*. The expected value of this variance for the state |y) is given by 
(p (A — E[A] "|y). 

We now give a mathematical proof that there is a theoretical lower limit for 
the product of the uncertainties in any two physical quantities, and we start by 
proving a result similar to the Schwarz inequality. Let |u) and |v) be any two state 
vectors and let 2 be any real scalar. Then consider the vector |w) = |u) + A|v) and, 
in particular, note that 


0 < (wl w) = (ul u) + A((u| 0) + (vu) + 270 | 0). 


This is a quadratic inequality in 2 and therefore the quadratic equation formed 
by equating the RHS to zero must have no real roots. The coefficient of 1 is 
((u|v) + (v|u)) = 2 Re (u|v) and its square is thus > 0. The condition for no real 
roots of the quadratic is therefore 


0 <((ulv) + (vl u))? < 4(ulu) (vv). (19.36) 


This result will now be applied to state vectors constructed from |p), the state 
vector of the particular system for which we wish to establish a relationship be- 
tween the uncertainties in the two physical variables corresponding to (Hermitian) 
operators A and B. We take 


\u) =(A—E[A])|p) and |v) = i(B— E[B])|y). (19.37) 
Then 
(ul u) = (p (A — E[A] 7 lp) = (AY, 
(v|v) = (p |(B — E[B] lp) = (ABY. 
Further, 


(u|v) = (p|(A— E[A])i(B — E[B])| yp) 
= i(y|AB|p) —iE[A](p| Bly) —iE[B](y| Alp) + iE[AJE[B] (yp |p) 
= i(p| AB | y) — iE[AJE[B]. 
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In the second line, we have moved expectation values, which are purely numbers, 
out of the inner products and used the normalisation condition (p|y) = 1. 
Similarly 


(v|u) = —i(p | BA |p) + iE [A]E[B]. 
Adding these two results gives 
(u|v) + (v|u) = i(p | AB — BAly), 
and substitution into (19.36) yields 
0 < (i(p| AB — BA |p). < 4(AA)P(ABY? 


At first sight, the middle term of this inequality might appear to be negative, but 
this is not so. Since A and B are Hermitian, AB— BA is anti-Hermitian, as is easily 
demonstrated. Since i is also anti-Hermitian, the quantity in the parentheses in 
the middle term is real and its square non-negative. Rearranging the equation and 
expressing it in terms of the commutator of A and B gives the generalised form 
of the Uncertainty Principle. For any particular state |y) of a system, this provides 
the quantitative relationship between the minimum value that the product of the 
uncertainties in A and B can have and the expectation value, in that state, of 
their commutator, 


(AA)(ABY > 5] (p| [4,B] |v) ?. (19.38) 


Immediate observations include the following: 


(i) If A and B commute there is no absolute restriction on the accuracy with 
which the corresponding physical quantities may be known. That is not to 
say that AA and AB will always be zero, only that they may be. 

(ii) If the commutator of A and B is a constant, k # 0, then the RHS of 
equation (19.38) is necessarily equal to xIKP, whatever the form of |), 
and it is not possible to have AA = AB = 0. 

(iii) Since the RHS depends upon |y), it is possible, even for two operators 
that do not commute, for the lower limit of (AA)?(AB)? to be zero. This 
will occur if the commutator [ A, B] is itself an operator whose expectation 
value in the particular state |y) happens to be zero. 


To illustrate the third of these, we might consider the components of angular 
momentum discussed in the previous subsection. There, in equation (19.27), we 
found that the commutator of the operators corresponding to the x- and y- 
components of angular momentum is non-zero; in fact, it has the value ihL,. 
This means that if the state |y) of a system happened to be such that (y|L.|y) = 0, 
as it would if, for example, it were the eigenstate of L., |y) = |/,0), then there 
would be no fundamental reason why the physical values of both L, and L, 
should not be known exactly. Indeed, if the state were spherically symmetric, and 
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hence formally an eigenstate of L? with / = 0, all three components of angular 
momentum could be (and are) known to be zero. 


> Working in one dimension, show that the minimum value of the product Ap, x Ax for 
a particle is 5h. Find the form of the wavefunction that attains this minimum value for a 


particle whose expectation values for position and momentum are X and p, respectively. 


We have already seen, in (19.23) that the commutator of p, and x is —ih, a constant. 
Therefore, irrespective of the actual form of |y), the RHS of (19.38) is zh? (see observation 
(ii) above). Thus, since all quantities are positive, taking the square roots of both sides of 
the equation shows directly that 


Apx x Ax > $h. 
Returning to the derivation of the Uncertainty Principle, we see that the inequality becomes 
an equality only when 
((u|v) + (vf u)P = 4(u|u) (vv). 
The RHS of this equality has the value 4||u||*||v||? and so, by virtue of Schwarz’s inequality, 
we have 
lull? lol]? = (uo) + lw) 
< (I(ulv)| + (vu)? 
< (ull lol] + [lel (lel)? 
= Alu? lo’. 
Since the LHS is less than or equal to something that has the same value as itself, all of 
the inequalities are, in fact, equalities. Thus (u|v) = |u| |||], showing that |u) and |v) are 
parallel vectors, i.e. |v) = |v) for some scalar pu. 


We now transform this condition into a constraint that the wavefunction yp = p(x) 
must satisfy. Recalling the definitions (19.37) of |v) and |v) in terms of |p), we have 


(“ing r) pue= sy 
dx 


d 1 
H+ =[ulx—X)—iply =0. 
‘ eee W(x—xXP  ipx 
The IF for this equation is exp — |, giving 
2h h 


which, in turn, leads to 


p(x) = Aexp — exp (®) : 


From this it is apparent that the minimum uncertainty product Ap, x Ax is obtained when 
the probability density |y(x)|? has the form of a Gaussian distribution centred on x. The 
value of y is not fixed by this consideration and it could be anything (positive); a large 
value for yz would yield a small value for Ax but a correspondingly large one for Ap,. < 
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19.2.3 Annihilation and creation operators 


As a final illustration of the use of operator methods in physics we consider their 
application to the quantum mechanics of a simple harmonic oscillator (s.h.o.). 
Although we will start with the conventional description of a one-dimensional 
oscillator, using its position and momentum, we will recast the description in 
terms of two operators and their commutator and show that many important 
conclusions can be reached from studying these alone. 

The Hamiltonian for a particle of mass m with momentum p moving in a 
one-dimensional parabolic potential V(x) = skx? is 


2 2 
P 1 >_P 1 ow 
Wa fo wpe fate 
ma om 
where its classical frequency of oscillation is given by m? = k/m. We recall that 
the corresponding operators, p and x, do not commute and that [p,x] = —ih. 


In analogy with the ladder operators used when discussing angular momentum, 
we define two new operators: 


— ud and At=,/"@x~ iz 


a ./2mo 2 2m (19.39) 


Since both x and p are Hermitian, A and A‘ are Hermitian conjugates, though 
neither is Hermitian and they do not represent physical quantities that can be 
measured. 

Now consider the two products AA and AA?: 


om 
Ill 
| 


mo ipx  ixp p Ai A Rh 
AA= 2 = = 
5 OS Sane Gg 
: : 2 e 
mo ipx  ixp Dp Ai A hk 
AAI = x? 4 5-4 =—4-=[px]=—+-2. 
ey = ot omy ot eo 
From these it follows that 
H = 50(AtA + AAt) (19.40) 
and that [A, At] =h. (19.41) 
Further, 
[H,A] = | 5@(A‘A + AA‘), A] 
= 5 (410+ [At,A] A+ 4[At, A] +041) 
= 40(—hA — Ah) = —hod. (19.42) 
Similarly, [H,A'] = hot (19.43) 


Before we apply these relationships to the question of the energy spectrum of 
the s.h.o., we need to prove one further result. This is that if B is an Hermitian 
operator then (y | B?| wp) > 0 for any |p). The proof, which involves introducing 
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an arbitrary complete set of orthonormal base states |¢;) and using equation 
(19.11), is as follows: 


(p|B? |p) =(p|Bx 1 x Bly) 


= >_ (pw! Bldi)(¢i| Bly) 
= Sop B\gi) ((¢:| Blp)*)” 
= S-(p| Bldi) ((p| Bt |g)” 


=> (p|Bldi)(y|B\¢i)", since B is Hermitian, 
i 

= S51 (y|Bldi) ? = 0. 
i 


We note, for future reference, that the Hamiltonian H for the s.h.o. is the sum of 
two terms each of this form and therefore conclude that (y|H|y) > 0 for all |p). 


The energy spectrum of the simple harmonic oscillator 

Let the normalised ket vector |n) (or |E,)) denote the nth energy state of the s.h.o. 
with energy E,. Then it must be an eigenstate of the (Hermitian) Hamiltonian H 
and satisfy 

H|n) = E,|n) with (min) = mn. 
Now consider the state A|n) and the effect of H upon it: 

HA|n) = AH|n) — h@A|n), using (19.42), 

= AE,|n) — h@A|n) 

= (E, — ha)Ajn). 
Thus Aln) is an eigenstate of H corresponding to energy E, — hw and must be 
some multiple of the normalised ket vector |E, — ha), ice. 


A| En) = Aln) = cy|En — ho), 


where c, is not necessarily of unit modulus. Clearly, A is an operator that 
generates a new state that is lower in energy by ha; it can thus be compared to 
the operator D, which has a similar effect in the context of the z-component of 
angular momentum. Because it possesses the property of reducing the energy of 
the state by iw, which, as we will see, is one quantum of excitation energy for the 
oscillator, the operator A is called an annihilation operator. Repeated application 
of A, m times say, will produce a state whose energy is miw lower than that of 
the original: 


A" Ey) = CnCp—1 ++ * Cr—m41|En — mio). (19.44) 
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In a similar way it can be shown that A’ parallels the operator U of our angular 
momentum discussion and creates an additional quantum of energy each time it 
is applied: 


(AT)"|En) = dndnsi +++ dn4m—1|En + mh). (19.45) 


It is therefore known as a creation operator. 

As noted earlier, the expectation value of the oscillator’s energy operator 
(p|H|p) must be non-negative, and therefore it must have a lowest value. Let 
this be Eo, with corresponding eigenstate |0). Since the energy-lowering property 
of A applies to any eigenstate of H, in order to avoid a contradiction we must 
have that A|0) = |@). It then follows from (19.40) that 


H|0) = $a@(AtA + AAt)|0) 
= 50A'A|0) + 50@(ATA + h)|0), using (19.41), 
0+0+ 5ho|0). (19.46) 


ll 


This shows that the commutator structure of the operators and the form of the 
Hamiltonian imply that the lowest energy (its ground-state energy) is sho; this 
is a result that has been derived without explicit reference to the corresponding 
wavefunction. This non-zero lowest value for the energy, known as the zero-point 
energy of the oscillator, and the discrete values for the allowed energy states 
are quantum-mechanical in origin; classically such an oscillator could have any 
non-negative energy, including zero. 

Working back from this result, we see that the energy levels of the s.h.o. are 
sho, sho, $ho,... (m+ 5)hio,..., and that the corresponding (unnormalised) 
ket vectors can be written as 


0), ATO), = (ATP10), ss (ATY™10), 


This notation, and elaborations of it, are often used in the quantum treatment of 
classical fields such as the electromagnetic field. Thus, as the reader should verify, 
A(At)3A?ATA(AT)4]0) is a state with energy sho, whilst A(A‘)3A4°AtA(A‘)4|0) is 
not a physical state at all. 


The normalisation of the eigenstates 
In order to make quantitative calculations using the previous results we need to 
establish the values of the c, and d, that appear in equations (19.44) and (19.45). 
To do this, we first establish the operator recurrence relation 


AM ATY™ = A” At)" 4 + maa” "Ary! (19.47) 
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The proof, which makes repeated use of [ A, AT] = h, is as follows: 
Am At)" = A" 4g atatye 
=A™"(4tA + Ay(Aty| 
= A™ TAT A(ATY" + AMAT 
= A" AN ATA + hy(Atyn™ + han l(Atyn! 
= AM (ATP ACAT "=? 4 AM! ATA(AT "2 + RAM (AT) 
= A™'WATP(ATA + hy(Aty + 2A" "(At yr 


= Ar (AT)"A 4 mhA™ (Ary, 
Now we take the expectation values in the ground state |0) of both sides of 
this operator equation and note that the first term on the RHS is zero since it 
contains the term A|0). The non-vanishing terms are 
(0| A™(AT)" |0) = mh(0| A™-!(AT "| 0). 

The LHS is the square of the norm of (A‘)”|0), and, from equation (19.45), it is 
equal to 

Idol*|dil? - += |dm—1|7 (0| 0). 
Similarly, the RHS is equal to 

mh |do|"|d1|? - = -|dm—zl7 (0| 0). 


It follows that |dm—1|? = mh and, taking all coefficients as real, dy = ./(m+ Ih. 
Thus the correctly normalised state of energy (n + 5)h, obtained by repeated 
application of At to the ground state, is given by 

(aty" 


ee (nl hn)t/2 [0). (9-48) 


To evaluate the c,, we note that, from the commutator of A and Af, 
[ A, A" | |n) = AA‘|n) — A‘ Ajn) 
Aln) = (n+ DRA |n + 1) —c, At |n— 1) 
= Va + Ihengs In) — en/nh |n), 


which has the obvious solution c, = ./nh. To summarise: 


C,=Jnh and d,=/(n+ Df. (19.49) 


We end this chapter with another worked example. This one illustrates how 


the operator formalism that we have developed can be used to obtain results 
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that would involve a number of non-trivial integrals if tackled using explicit 
wavefunctions. 


> Given that the first-order change in the ground-state energy of a quantum system when 
it is perturbed by a small additional term H' in the Hamiltonian is (0|H"'\0), find the first- 
order change in the energy of a simple harmonic oscillator in the presence of an additional 
potential V'(x) = Ax? + px*. 


From the definitions of A and A‘, equation (19.39), we can write 


Ll 


(A+Al)) => Wao agaist 
(2mm) 


1 
./2ma (2mw)3/2 


We now compute successive values of (A + At)" |0) for n = 1,2,3,4, remembering that 
A|n) = V/nh|n — 1) and Al |n) =. /(+Dhi|n+1): 


(A+ 4‘) |0) =0+h'/ |1), 
(A+ A‘)? |0) = AIO) + 2h |2), 
(A + AT)? 0) = 0+ WY? |1) + 207? [1) + 6? |3) 
= 3h? |1) + Jon"? |3), 
(A + At)* 0) = 3A? |0) + /18 h? |2) + 18 h? 2) + 24h? |4) 


To find the energy shift we need to form the inner product of each of these state vectors 
with |0). But |0) is orthogonal to all |n) if n 4 0. Consequently, the term (0|(A +A‘)? |0) 
in the expectation value is zero, and in the expression for (0|(A + A‘)*|0) only the first 
term is non-zero; its value is 3h?. The perturbation energy is thus given by 


3h? 
(2ma)?* 
It could have been anticipated on symmetry grounds that the expectation of 4x3, an 
odd function of x, would be zero, but the calculation gives this result automatically. The 


contribution of the quadratic term in the perturbation would have been much harder to 
anticipate! < 


(A+ At. 


x= 


(0| H’|0) = 


19.3 Exercises 


19.1 Show that the commutator of two operators that correspond to two physical 
observables cannot itself correspond to another physical observable. 
19.2 By expressing the operator L,, corresponding to the z-component of angular 


momentum, in spherical polar coordinates (r,0,¢), show that the angular mo- 
mentum of a particle about the polar axis cannot be known at the same time as 
its azimuthal position around that axis. 

19.3 In quantum mechanics, the time dependence of the state function |p) of a system 
is given, as a further postulate, by the equation 


ae) 
ih—|p) = Hly), 
ot 


where H is the Hamiltonian of the system. Use this to find the time dependence 
of the expectation value (A) of an operator A that itself has no explicit time 
dependence. Hence show that operators that commute with the Hamiltonian 
correspond to the classical ‘constants of the motion’. 
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19.4 


19.5 


19.6 


19.7 


19.8 


For a particle of mass m moving in a one-dimensional potential V(x), prove 
Ehrenfest’s theorem: 


is) (Maggs) _ 


Show that the Pauli matrices 


01 0 =i 1 0 
s.=4h({ 6 s,=4n( § 0 ie s.=4n( 4 4) 


which are used as the operators corresponding to intrinsic spin of sh in non- 


relativistic quantum mechanics, satisfy $2 = S$? = S$? = zh | |, and have the 
same commutation properties as the components of orbital angular momentum. 
Deduce that any state |y) represented by the column vector (a, b)' is an eigenstate 
of S? with eigenvalue 3h? /4. 

Find closed-form expressions for cosC and sin C, where C is the matrix 


1 1 
c-(1 4). 
Demonstrate that the ‘expected’ relationships 


cos’’C+sin?C =I and sin2C =2sinCcosC 


are valid. 
Operators A and B anticommute. Evaluate (A + B)*" for a few values of n and 
hence propose an expression for c,, in the expansion 


(A+ By" = 3 Co Aan Br. 


Prove your proposed formula for general values of n, using the method of 
induction. 
Show that 


cos(A + B) sy oo yy A2"-?" B?", 


n=0 r=0 
where the d,, are constants whose values you should determine. 
By taking as A the matrix A = : ; ). confirm that your answer is 


consistent with that obtained in exercise 19.5. 

Expressed in terms of the annihilation and creation operators A and A? discussed 
in the text, a system has an unperturbed Hamiltonian Hp = hwA‘A. The system 
is disturbed by the addition of a perturbing Hamiltonian H, = ghaw(A + A‘), 
where g is real. Show that the effect of the perturbation is to move the whole 
energy spectrum of the system down by gho. 

For a system of N electrons in their ground state |0), the Hamiltonian is 


N42 2 2 N 
y Py, + Pry + Pen ss 
H= => or + V(Xn, Yns Zn) 


n=1 n=1 
Show that | p,,x,] = —2ihp,,, and hence that the expectation value of the 
double commutator [[x,H],x], where x = pels Xn, is given by 
Nh2 


(0| [[x,H],x] 10) = —— 
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19.9 


19.10 


Now evaluate the expectation value using the eigenvalue properties of H, namely 
H|r) = E,|r), and deduce the sum rule for oscillation strengths, 


SE, — Eye xo) = SE, 


r=0 
By considering the function 
F(A) = exp(AA)B exp(—/A), 


where A and B are linear operators and 4 is a parameter, and finding its 
derivatives with respect to A, prove that 


SIATABI + SATA LAB] +0. 


e*Be 4 =B+[A,B]+ a 


Use this result to express 


iL yO —iL0 
exp ae Ly exp i 


as a linear combination of the angular momentum operators L,, Ly and L;. 
For a system containing more than one particle, the total angular momentum J 
and its components are represented by operators that have completely analogous 
commutation relations to those for the operators for a single particle, ic. J? has 
eigenvalue j(j + 1)h? and J, has eigenvalue m;h for the state |j,m;). The usual 
orthonormality relationship (j’,m| j,mj) = 57; On ‘mj is also valid. 


A system consists of two (distinguishable) particles A and B. Particle A is in 
an / = 3 state and can have state functions of the form |A,3,m,), whilst B is 
in an / = 2 state with possible state functions |B,2,mg). The range of possible 
values for j is 3 —2| < j < |3+2|, 1e. 1 < j < 5, and the overall state function 
can be written as 


lim) =  S> Cinthng| 4.3.4) |B, 2,m). 


m4a+mp=mj 


The numerical coefficients Cite are known as Clebsch—Gordon coefficients. 
Assume (as can be shown) that the ladder operators U(AB) and D(AB) for 
the system can be written as U(A) + U(B) and D(A) + D(B), respectively, and 
that they lead to relationships equivalent to (19.34) and (19.35) with 7 replaced 
by j and m by mj. 
(a) Apply the operators to the (obvious) relationship 
|AB, 5,5) = |A, 3, 3) |B, 2, 2) 


to show that 
|AB, 5,4) = \/ 1A, 3, 2) |B, 2,2) + V4 |A, 3, 3) |B, 2, 1). 


(b) Find, to within an overall sign, the real coefficients c and d in the expansion 


|AB, 4,4) = c|A, 3,2) |B, 2,2) + djA, 3, 3) |B, 2, 1) 


by requiring it to be orthogonal to |AB,5,4). Check your answer by considering 
U(AB)|AB, 4,4). 

(c) Find, to within an overall sign, and as efficiently as possible, an expression 
for |AB,4,—3) as a sum of products of the form |A, 3,m,) |B, 2, mg). 
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19.1 
19.3 


19.5 


19.7 


19.9 


19.4 Hints and answers 


Show that the commutator is anti-Hermitian. 

Use the Hermitian conjugate of the given equation to obtain the time dependence 
of (p|. The rate of change of (p|Aly) is i(p|[H,A]|p). Note that [H,p.] = 
[V, px] and [H,x] = [ p2,x] /2m. 

Show that C? = 21. 


cos = 60s V3 ( . : 1 sinC = my ( ; =, ). 
Express the total Hamiltonian in terms of B = A+ gI and determine the value 
of |B, Bt]. 
Show that, if F” is the nth derivative of F(A), then F"*!) = [| A, F |. Use a Taylor 
series in 4 to evaluate F(1), using derivatives evaluated at 14 = 0. Successively 
reduce the level of nesting of each multiple commutator by using the result of 
evaluating the previous term. The given expression reduces to cos 0 Ly — sin 6 L;. 
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Partial differential equations: 
general and particular solutions 


In this chapter and the next the solution of differential equations of types 
typically encountered in the physical sciences and engineering is extended to 
situations involving more than one independent variable. A partial differential 
equation (PDE) is an equation relating an unknown function (the dependent 
variable) of two or more variables to its partial derivatives with respect to 
those variables. The most commonly occurring independent variables are those 
describing position and time, and so we will couch our discussion and examples 
in notation appropriate to them. 

As in other chapters we will focus our attention on the equations that arise 
most often in physical situations. We will restrict our discussion, therefore, to 
linear PDEs, i.e. those of first degree in the dependent variable. Furthermore, we 
will discuss primarily second-order equations. The solution of first-order PDEs 
will necessarily be involved in treating these, and some of the methods discussed 
can be extended without difficulty to third- and higher-order equations. We shall 
also see that many ideas developed for ordinary differential equations (ODEs) 
can be carried over directly into the study of PDEs. 

In this chapter we will concentrate on general solutions of PDEs in terms 
of arbitrary functions and the particular solutions that may be derived from 
them in the presence of boundary conditions. We also discuss the existence and 
uniqueness of the solutions to PDEs under given boundary conditions. 

In the next chapter the methods most commonly used in practice for obtaining 
solutions to PDEs subject to given boundary conditions will be considered. These 
methods include the separation of variables, integral transforms and Green’s 
functions. This division of material is rather arbitrary and has been made only 
to emphasise the general usefulness of the latter methods. In particular, it will 
be readily apparent that some of the results of the present chapter are in 
fact solutions in the form of separated variables, but arrived at by a different 
approach. 
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20.1 Important partial differential equations 


Most of the important PDEs of physics are second-order and linear. In order to 
gain familiarity with their general form, some of the more important ones will 
now be briefly discussed. These equations apply to a wide variety of different 
physical systems. 

Since, in general, the PDEs listed below describe three-dimensional situations, 
the independent variables are r and t, where r is the position vector and t is 
time. The actual variables used to specify the position vector r are dictated by the 
coordinate system in use. For example, in Cartesian coordinates the independent 
variables of position are x, y and z, whereas in spherical polar coordinates they 
are r, 0 and ¢. The equations may be written in a coordinate-independent manner, 
however, by the use of the Laplacian operator V?. 


20.1.1 The wave equation 
The wave equation 


2 
Vu= 1 Oru 


~ 2 6p 


(20.1) 


describes as a function of position and time the displacement from equilibrium, 
u(r, t), of a vibrating string or membrane or a vibrating solid, gas or liquid. The 
equation also occurs in electromagnetism, where u may be a component of the 
electric or magnetic field in an elecromagnetic wave or the current or voltage 
along a transmission line. The quantity c is the speed of propagation of the waves. 


> Find the equation satisfied by small transverse displacements u(x,t) of a uniform string of 
mass per unit length p held under a uniform tension T, assuming that the string is initially 
located along the x-axis in a Cartesian coordinate system. 


Figure 20.1 shows the forces acting on an elemental length As of the string. If the tension 
T in the string is uniform along its length then the net upward vertical force on the 
element is 


AF = T sin 0, — T sin 0. 


Assuming that the angles 0; and @) are both small, we may make the approximation 
sin@ = tan. Since at any point on the string the slope tan? = du/0x, the force can be 
written 

du(x + Ax,t)  du(x,t) 07 u(x, t) 


AF = T ax ax ~T axe Ax, 


where we have used the definition of the partial derivative to simplify the RHS. 

This upward force may be equated, by Newton’s second law, to the product of the 
mass of the element and its upward acceleration. The element has a mass p As, which is 
approximately equal to p Ax if the vibrations of the string are small, and so we have 

07u(x, t) 07 u(x, t) 


ap =T re Ax. 
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x x+Ax “x 


Figure 20.1 The forces acting on an element of a string under uniform 
tension T. 


Dividing both sides by Ax we obtain, for the vibrations of the string, the one-dimensional 
wave equation 
au 1 @u 


Ox? ce? At?’ 


where c? = T/p. 4 


The longitudinal vibrations of an elastic rod obey a very similar equation to 
that derived in the above example, namely 


Ou _ p Oru 
ax? E Ot?’ 
here p is the mass per unit volume and E is Young’s modulus. 

The wave equation can be generalised slightly. For example, in the case of the 
vibrating string, there could also be an external upward vertical force f(x, t) per 
unit length acting on the string at time ¢t. The transverse vibrations would then 
satisfy the equation 

a Ou 


u 
5) t f(%0 = pa 


which is clearly of the form ‘upward force per unit length = mass per unit length 
x upward acceleration’. 

Similar examples, but involving two or three spatial dimensions rather than one, 
are provided by the equation governing the transverse vibrations of a stretched 
membrane subject to an external vertical force density ais y,t), 


Ou Ou 
T (3 +4 =) + f(x y,t) = p(x, jee _ 
where p is the mass per unit area of the membrane and T is the tension. 
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20.1.2 The diffusion equation 


The diffusion equation 


_ Ou 


2 
= _ 20.2 
KV-u at ( ) 


describes the temperature u in a region containing no heat sources or sinks; it 
also applies to the diffusion of a chemical that has a concentration u(r, t). The 
constant x is called the diffusivity. The equation is clearly second order in the 
three spatial variables, but first order in time. 


> Derive the equation satisfied by the temperature u(r, t) at time t for a material of uniform 
thermal conductivity k, specific heat capacity s and density p. Express the equation in 
Cartesian coordinates. 


Let us consider an arbitrary volume V lying within the solid and bounded by a surface S 
(this may coincide with the surface of the solid if so desired). At any point in the solid 
the rate of heat flow per unit area in any given direction f is proportional to minus the 
component of the temperature gradient in that direction and so is given by (—kVu)-f. The 
total flux of heat out of the volume V per unit time is given by 


2 = | [ev as 
= [[[¥ Caviar, (20.3) 


where Q is the total heat energy in V at time ¢ and fi is the outward-pointing unit normal 
to S; note that we have used the divergence theorem to convert the surface integral into 
a volume integral. 

We can also express Q as a volume integral over V, 


o- fff smav. 


and its rate of change is then given by 


dQ ou 
Tie eae > dV, (20.4) 


where we have taken the derivative with respect to time inside the integral (see section 5.12). 
Comparing (20.3) and (20.4), and remembering that the volume V is arbitrary, we obtain 
the three-dimensional diffusion equation 


ou 
2 
kVu=—, 
Ot 


where the diffusion coefficient « = k/(sp). To express this equation in Cartesian coordinates, 
we simply write V7 in terms of x, y and z to obtain 


: o7u de Oru rf au\ — du ed 
ox? ° dy? © az?) at’ 


The diffusion equation just derived can be generalised to 


ou 
2 : =e 
kV-ut+ f(r,t) = pz 
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The second term, f(r,t), represents a varying density of heat sources throughout 
the material but is often not required in physical applications. In the most general 
case, k, s and p may depend on position r, in which case the first term becomes 
V - (kVu). However, in the simplest application the heat flow is one-dimensional 
with no heat sources, and the equation becomes (in Cartesian coordinates) 


20.1.3 Laplace’s equation 
Laplace’s equation, 
Vu =0, (20.5) 


may be obtained by setting du/ét = 0 in the diffusion equation (20.2), and 
describes (for example) the steady-state temperature distribution in a solid in 
which there are no heat sources — 1.e. the temperature distribution after a long 
time has elapsed. 

Laplace’s equation also describes the gravitational potential in a region con- 
taining no matter or the electrostatic potential in a charge-free region. Further, it 
applies to the flow of an incompressible fluid with no sources, sinks or vortices; 
in this case u is the velocity potential, from which the velocity is given by v = Vu. 


20.1.4 Poisson’s equation 


Poisson’s equation, 
V-u = p(n), (20.6) 


describes the same physical situations as Laplace’s equation, but in regions 
containing matter, charges or sources of heat or fluid. The function p(r) is 
called the source density and in physical applications usually contains some 
multiplicative physical constants. For example, if u is the electrostatic potential 
in some region of space, in which case p is the density of electric charge, then 
V7u = —p(r)/e0, where eo is the permittivity of free space. Alternatively, u might 
represent the gravitational potential in some region where the matter density is 
given by p; then Vu = 42Gp(r), where G is the gravitational constant. 


20.1.5 Schredinger’s equation 


The Schrodinger equation 


ne , 
Vu LV(rju = in, (20.7) 
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describes the quantum mechanical wavefunction u(r, t) of a non-relativistic particle 
of mass m; fi is Planck’s constant divided by 27. Like the diffusion equation it is 
second order in the three spatial variables and first order in time. 


20.2 General form of solution 


Before turning to the methods by which we may hope to solve PDEs such as 
those listed in the previous section, it is instructive, as for ODEs in chapter 14, to 
study how PDEs may be formed from a set of possible solutions. Such a study 
can provide an indication of how equations obtained not from possible solutions 
but from physical arguments might be solved. 

For definiteness let us suppose we have a set of functions involving two 
independent variables x and y. Without further specification this is of course a 
very wide set of functions, and we could not expect to find a useful equation that 
they all satisfy. However, let us consider a type of function u;(x, y) in which x and 
y appear in a particular way, such that u; can be written as a function (however 
complicated) of a single variable p, itself a simple function of x and y. 

Let us illustrate this by considering the three functions 


(x,y) =x" +4(Qx"y +? +0), 
u2(x, y) = sin x” cos 2y + cos x’ sin 2y, 
x? + 2y +2 

u3(x,y) = 3x2 +6y +5 
These are all fairly complicated functions of x and y and a single differential 
equation of which each one is a solution is not obvious. However, if we observe 
that in fact each can be expressed as a function of the variable p = x? + 2y alone 
(with no other x or y involved) then a great simplification takes place. Written 
in terms of p the above equations become 

wi(x,y) = (x? + 2yP +4 = p? +4 = filp), 

uo(x, y) = sin(x* + 2y) = sin p = fr(p), 

— (QP +2y)4+2 — pt2 _, 


Let us now form, for each u;, the partial derivatives Gu;/0x and du;/dy. In each 


case these are (writing both the form for general p and the one appropriate to 
our particular case, p = x? + 2y) 


Ou; — dafi(p) Op _ 3 


Se = = ons, 
Ox dp 0x 
Au, dflp)a 
a Filp) Op _ of, 
oy dp oy 
for i = 1, 2, 3. All reference to the form of f; can be eliminated from these 
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equations by cross-multiplication, obtaining 


Op ou; = Op Ou; 


dy dx dx dy’ 
or, for our specific form, p = x* + 2y, 


OU; ateet OU; 


= ; 20.8 
ox oy 08) 
It is thus apparent that not only are the three functions 1, u2 u3 solutions of the 
PDE (20.8) but so also is any arbitrary function f(p) of which the argument p has 


the form x? + 2y. 


20.3. General and particular solutions 


In the last section we found that the first-order PDE (20.8) has as a solution any 
function of the variable x? + 2y. This points the way for the solution of PDEs 
of other orders, as follows. It is not generally true that an nth-order PDE can 
always be considered as resulting from the elimination of n arbitrary functions 
from its solution (as opposed to the elimination of n arbitrary constants for an 
nth-order ODE, see section 14.1). However, given specific PDEs we can try to 
solve them by seeking combinations of variables in terms of which the solutions 
may be expressed as arbitrary functions. Where this is possible we may expect n 
combinations to be involved in the solution. 

Naturally, the exact functional form of the solution for any particular situation 
must be determined by some set of boundary conditions. For instance, if the PDE 
contains two independent variables x and y then for complete determination of 
its solution the boundary conditions will take a form equivalent to specifying 
u(x, y) along a suitable continuum of points in the xy-plane (usually along a line). 

We now discuss the general and particular solutions of first- and second- 
order PDEs. In order to simplify the algebra, we will restrict our discussion 
to equations containing just two independent variables x and y. Nevertheless, 
the method presented below may be extended to equations containing several 
independent variables. 


20.3.1 First-order equations 


Although most of the PDEs encountered in physical contexts are second order 
(i.e. they contain 0?u/0x? or 07u/Adxéy, etc.), we now discuss first-order equations 
to illustrate the general considerations involved in the form of the solution and 
in satisfying any boundary conditions on the solution. 

The most general first-order linear PDE (containing two independent variables) 
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is of the form 

Als) + BCE) + C(x. )u = ROY) (20.9) 

XV) x,y By x, yu = R(x, y), F 

where A(x, y), B(x, y), C(x,y) and R(x, y) are given functions. Clearly, if either 
A(x,y) or B(x, y) is zero then the PDE may be solved straightforwardly as a 
first-order linear ODE (as discussed in chapter 14), the only modification being 
that the arbitrary constant of integration becomes an arbitrary function of x or y 
respectively. 


> Find the general solution u(x, y) of 


6 
age +3u=x’. 
0x 


Dividing through by x we obtain 


ou 3u 
a Toe ee 


Ox x 


which is a linear equation with integrating factor (see subsection 14.2.4) 


exp (/ 2 ax) = exp(3Inx) = x°. 
x 


Multiplying through by this factor we find 


> 


(x3u) = x*, 


D 


x 


which, on integrating with respect to x, gives 


x 
su= = +f), 


where f(y) is an arbitrary function of y. Finally, dividing through by x*, we obtain the 
solution 
x | fy) 
x= z+>.< 
u(x, y) st 
When the PDE contains partial derivatives with respect to both independent 
variables then, of course, we cannot employ the above procedure but must seek 
an alternative method. Let us for the moment restrict our attention to the special 
case in which C(x, y) = R(x, y) = 0 and, following the discussion of the previous 
section, look for solutions of the form u(x, y) = f(p) where p is some, at present 
unknown, combination of x and y. We then have 
du _ df(p) Op 


ox dp ox’ 
ou _ df(p) Op 
éy dp 6y’ 
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which, when substituted into the PDE a give 


af(p) 


=0. 
oy “dp 


A(x, yee B(x, ye 


This removes all reference to the actual form of the function f(p) since for 
non-trivial p we must have 


Op 
A(x, Way ee B(x xy =o. (20.10) 


Let us now consider the necessary condition for f(p) to remain constant as x 
and y vary; this is that p itself remains constant. Thus for f to remain constant 
implies that x and y must vary in such a way that 


a 
dp = — dx + —dy =0. (20.11) 
oy 
The forms of (20.10) and (20.11) are very alike and become the same if we 
require that 
dx dy 
A(x,y) — BOx, y)” 


By integrating this expression the form of p can be found. 


(20.12) 


ou ou 
hae a =), (20.13) 


find (i) the solution that takes the value 2y +1 on the line x = 1, and (ii) a solution that 
has the value 4 at the point (1,1). 


If we seek a solution of the form u(x, y) = f(p), we deduce from (20.12) that u(x, y) will 
be constant along lines of (x, y) that satisfy 


dx dy 


x —2y’ 
which on integrating gives x = cy—!/?. Identifying the constant of integration c with p'/? 
(to avoid fractional powers), we conclude that p = x’y. Thus the general solution of the 
PDE (20.13) is 


u(x, y) = f(y), 
where f is an arbitrary function. 

We must now find the particular solutions that obey each of the imposed boundary 
conditions. For boundary condition (i) a little thought shows that the particular solution 
required is 

u(x, y) = 2(x?y) +1 =2x?y +1. (20.14) 
For boundary condition (ii) some obviously acceptable solutions are 
u(x, y) = xy +3, 
u(x, y) = 4x°y, 
u(x, y) = 4. 
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Each is a valid solution (the freedom of choice of form arises from the fact that u 
is specified at only one point (1,1), and not along a continuum (say), as in boundary 
condition (i)). All three are particular examples of the general solution, which may be 
written, for example, as 


u(x, y) = xy +3 + g(x"y), 

where g = g(x*y) = g(p) is an arbitrary function subject only to g(1) = 0. For this 
example, the forms of g corresponding to the particular solutions listed above are g(p) = 0, 
g(p) = 3p — 3, g(p) =1—p. 4 

As mentioned above, in order to find a solution of the form u(x, y) = f(p) we 
require that the original PDE contains no term in u, but only terms containing 
its partial derivatives. If a term in u is present, so that C(x,y) # 0 in (20.9), 
then the procedure needs some modification, since we cannot simply divide out 
the dependence on f(p) to obtain (20.10). In such cases we look instead for 
a solution of the form u(x,y) = h(x, y)f(p). We illustrate this method in the 
following example. 


> Find the general solution of 


seh a ay (20.15) 
Ox oy 


du oh ate) op 
Ox th dp 6x’ 
ou Ooh nf ) é 

Ny) LOE 
oy oy ap oy" 


Substituting these expressions into the PDE (20.15) and rearranging, we obtain 


oh oh op pti) 
=0. 
(5 oy t) fort (*F i) “dpe 
The first factor in parentheses is just the original PDE with u replaced by h. Therefore, if 
h is any solution of the PDE, however simple, this term will vanish, to leave 


Op df) 
(+25)! aps 


from which, as in the previous case, we obtain 


From (20.11) and (20.12) we see that u(x, y) will be constant along lines of (x,y) that 
satisfy 

dx _ dy 

x: 1D? 
which integrates to give x = cexp(y/2). Identifying the constant of integration c with p we 
find p = xexp(—y/2). Thus the general solution of (20.15) is 

u(x, y) = h(x, y)f(xexp(—3y)), 

where f(p) is any arbitrary function of p and h(x, y) is any solution of (20.15). 
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If we take, for example, h(x, y) = exp y, which clearly satisfies (20.15), then the general 
solution is 


u(x, y) = (exp y)f(xexp(—3y)). 
Alternatively, h(x, y) = x? also satisfies (20.15) and so the general solution to the equation 
can also be written 
u(x, y) = x°g(xexp(—}y)), 
where g is an arbitrary function of p; clearly g(p) = f(p)/p’. < 


20.3.2 Inhomogeneous equations and problems 


Let us discuss in a more general form the particular solutions of (20.13) found 
in the second example of the previous subsection. It is clear that, so far as this 
equation is concerned, if u(x, y) is a solution then so is any multiple of u(x, y) or 
any linear sum of separate solutions u(x, y) + u(x, y). However, when it comes 
to fitting the boundary conditions this is not so. 

For example, although u(x, y) in (20.14) satisfies the PDE and the boundary 
condition u(1,y) = 2y + 1, the function u;(x,y) = 4u(x,y) = 8xy + 4, whilst 
satisfying the PDE, takes the value 8y+4 on the line x = 1 and so does not satisfy 
the required boundary condition. Likewise the function u(x, y) = u(x, y)+f1(x7y), 
for arbitrary f,, satisfies (20.13) but takes the value u2(1,y) = 2y +1+f1(y) on 
the line x = 1, and so is not of the required form unless f; is identically zero. 

Thus we see that when treating the superposition of solutions of PDEs two 
considerations arise, one concerning the equation itself and the other connected 
to the boundary conditions. The equation is said to be homogeneous if the fact 
that u(x, y) is a solution implies that Au(x, y), for any constant /, is also a solution. 
However, the problem is said to be homogeneous if, in addition, the boundary 
conditions are such that if they are satisfied by u(x, y) then they are also satisfied 
by Au(x,y). The last requirement itself is referred to as that of homogeneous 
boundary conditions. 

For example, the PDE (20.13) is homogeneous but the general first-order 
equation (20.9) would not be homogeneous unless R(x,y) = 0. Furthermore, 
the boundary condition (i) imposed on the solution of (20.13) in the previous 
subsection is not homogeneous though, in this case, the boundary condition 


u(x, y) =0 on the line y= 4x72 


would be, since u(x, y) = A(x?y — 4) satisfies this condition for any A and, being a 
function of x7y, satisfies (20.13). 

The reason for discussing the homogeneity of PDEs and their boundary condi- 
tions is that in linear PDEs there is a close parallel to the complementary-function 
and particular-integral property of ODEs. The general solution of an inhomo- 
geneous problem can be written as the sum of any particular solution of the 
problem and the general solution of the corresponding homogeneous problem (as 
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for ODEs, we require that the particular solution is not already contained in the 
general solution of the homogeneous problem). Thus, for example, the general 
solution of 

ou Ou 
—- * 5 + au = f(x,y), (20.16) 


Ox 


subject to, say, the boundary condition u(0, y) = g(y), is given by 
u(x, y) = v(x, y) + w(x, y), 


where v(x, y) is any solution (however simple) of (20.16) such that v(0, y) = g(y) 
and w(x, y) is the general solution of 


Ow ow 
zs 7 * 


— +aw =0, (20.17) 
Ox oy 


with w(0, y) = 0. If the boundary conditions are sufficiently specified then the only 
possible solution of (20.17) will be w(x, y) = 0 and v(x, y) will be the complete 
solution by itself. 

Alternatively, we may begin by finding the general solution of the inhomoge- 
neous equation (20.16) without regard for any boundary conditions; it is just the 
sum of the general solution to the homogeneous equation and a particular inte- 
gral of (20.16), both without reference to the boundary conditions. The boundary 
conditions can then be used to find the appropriate particular solution from the 
general solution. 

We will not discuss at length general methods of obtaining particular integrals 
of PDEs but merely note that some of those methods available for ordinary 
differential equations can be suitably extended § 


> Find the general solution of 


ou ou 
Vax By 


Hence find the most general particular solution (i) which satisfies u(x,0) = x and (ii) which 
has the value u(x, y) = 2 at the point (1,0). 


=o5 (20.18) 


This equation is inhomogeneous, and so let us first find the general solution of (20.18) 
without regard for any boundary conditions. We begin by looking for the solution of the 
corresponding homogeneous equation ((20.18) but with the RHS equal to zero) of the 
form u(x, y) = f(p). Following the same procedure as that used in the solution of (20.13) 
we find that u(x, y) will be constant along lines of (x, y) that satisfy 


dx dy xy? 
Gag. PT a gs 


Identifying the constant of integration c with p/2, we find that the general solution of the 


S See for example H. T. H. Piaggio, An Elementary Treatise on Differential Equations and their 
Applications (London: G. Bell and Sons, Ltd, 1954), pp. 175 ff. 
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homogeneous equation is u(x, y) = f(x? + y”) for arbitrary function f. Now by inspection 
a particular integral of (20.18) is u(x, y) = —3y, and so the general solution to (20.18) is 
u(x, y) = fe + y’) — 3y. 

Boundary condition (i) requires u(x,0) = f(x”) = x, ie. f(z) = z, and so the particular 

solution in this case is 
u(x, y) =x? + y? —3y. 

Similarly, boundary condition (ii) requires u(1,0) = f(1) = 2. One possibility is f(z) = 2z, 
and if we make this choice, then one way of writing the most general particular solution 
is 


u(x, y) = 2x” + 2y* — 3y + g(x? +9"), 
where g is any arbitrary function for which g(1) = 0. Alternatively, a simpler choice would 
be f(z) = 2, leading to 
u(x,y) =2—3y +9(x’ +y’). < 


Although we have discussed the solution of inhomogeneous problems only 
for first-order equations, the general considerations hold true for linear PDEs of 
higher order. 


20.3.3 Second-order equations 


As noted in section 20.1, second-order linear PDEs are of great importance in 
describing the behaviour of many physical systems. As in our discussion of first- 
order equations, for the moment we shall restrict our discussion to equations with 
just two independent variables; extensions to a greater number of independent 
variables are straightforward. 

The most general second-order linear PDE (containing two independent vari- 
ables) has the form 


Ou Ou Ou ou ou 
A— + B— 4+ C—4+D— 4+ E— 4+ Fu= R(x, y), 20.19 
Bo ae ey 0) 
where A, B,...,F and R(x, y) are given functions of x and y. Because of the nature 


of the solutions to such equations, they are usually divided into three classes, a 
division of which we will make further use in subsection 20.6.2. The equation 
(20.19) is called hyperbolic if B* > 4AC, parabolic if B? = 4AC and elliptic if 
B? < 4AC. Clearly, if A, B and C are functions of x and y (rather than just 
constants) then the equation might be of different types in different parts of the 
xy-plane. 

Equation (20.19) obviously represents a very large class of PDEs, and it is 
usually impossible to find closed-form solutions to most of these equations. 
Therefore, for the moment we shall consider only homogeneous equations, with 
R(x, y) = 0, and make the further (greatly simplifying) restriction that, throughout 
the remainder of this section, A, B,...,F are not functions of x and y but merely 
constants. 
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We now tackle the problem of solving some types of second-order PDE with 
constant coefficients by seeking solutions that are arbitrary functions of particular 
combinations of independent variables, just as we did for first-order equations. 

Following the discussion of the previous section, we can hope to find such 
solutions only if all the terms of the equation involve the same total number 
of differentiations, i.e. all terms are of the same order, although the number 
of differentiations with respect to the individual independent variables may be 
different. This means that in (20.19) we require the constants D, E and F to be 
identically zero (we have, of course, already assumed that R(x, y) is zero), so that 
we are now considering only equations of the form 


07u 07u 07u 
A B t =0, 20.20 
Ox? 35 Oxdy e dy? ( ) 


where A, B and C are constants. We note that both the one-dimensional wave 
equation, 
Oru 1 0*u 
62 2 OP 
and the two-dimensional Laplace equation, 
Cu Ou =, 
6x2 Oye” 
are of this form, but that the diffusion equation, 


Cu Ou 


“Oe Ot 
is not, since it contains a first-order derivative. 

Since all the terms in (20.20) involve two differentiations, by assuming a solution 
of the form u(x, y) = f(p), where p is some unknown function of x and y (or f), 
we may be able to obtain a common factor d?f(p)/dp as the only appearance of 
f on the LHS. Then, because of the zero RHS, all reference to the form of f can 
be cancelled out. 

We can gain some guidance on suitable forms for the combination p = p(x, y) 
by considering 6u/0x when u is given by u(x, y) = f(p), for then 

du _ df(p) op 
ax dp ax’ 
Clearly differentiation of this equation with respect to x (or y) will not lead to a 
single term on the RHS, containing f only as d?f(p)/dp?, unless the factor ép/dx 
is a constant so that 4*p/0x? and 0*p/Axéy are necessarily zero. This shows that 
p must be a linear function of x. In an exactly similar way p must also be a linear 
function of y, i.e. p = ax + by. 

If we assume a solution of (20.20) of the form u(x, y) = f(ax+by), and evaluate 
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the terms ready for substitution into (20.20), we obtain 
0 df 0 d 
ou a f(p) du _p f (Pp) 


Ox dp’ oy dp ’ 
Cu 2 a f(p) Pu pet) Ou Re @ f(p) 
axe dp? sy (si dp Cs dp”? 


which on substitution give 


(Aa* + Bab + Cb’) 0. (20.21) 


Cf(p) _ 
dp* 
This is the form we have been seeking, since now a solution independent of 
the form of f can be obtained if we require that a and b satisfy 


Aad? + Bab + Cb? = 0. 


From this quadratic, two values for the ratio of the two constants a and b are 


obtained, 
b/a = [—B + (B? —4AC)!""]/2€. 


If we denote these two ratios by 4; and A» then any functions of the two variables 
Pr=xthy, pro=xt+hy 


will be solutions of the original equation (20.20). The omission of the constant 
factor a from p; and po is of no consequence since this can always be absorbed 
into the particular form of any chosen function; only the relative weighting of x 
and y in p is important. 

Since p; and p2 are in general different, we can thus write the general solution 
of (20.20) as 


u(x, y) = f(x + Ay) + g(x + A2y), (20.22) 


where f and g are arbitrary functions. 

Finally, we note that the alternative solution d?f(p)/dp* = 0 to (20.21) leads 
only to the trivial solution u(x, y) = kx +ly +m, for which all second derivatives 
are individually zero. 


> Find the general solution of the one-dimensional wave equation 
Ou 1 eu 


This equation is (20.20) with A = 1, B = 0 and C = —1/c’, and so the values of 4; and Ay 
are the solutions of 


42 


A 
i-— =0, 
oe 
namely 4, = —c and A, = c. This means that arbitrary functions of the quantities 
pi=xX—ct, pP2=x+ct 
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will be satisfactory solutions of the equation and that the general solution will be 
u(x,t) = f(x —ct) + g(x + ct), (20.23) 
where f and g are arbitrary functions. This solution is discussed further in section 20.4. < 


The method used to obtain the general solution of the wave equation may also 
be applied straightforwardly to Laplace’s equation. 


> Find the general solution of the two-dimensional Laplace equation 


Following the established procedure, we look for a solution that is a function f(p) of 
p=x+Ay, where from (20.24) 4 satisfies 


1+7=0. 
This requires that 2 = +i, and satisfactory variables p are p = x tiy. The general solution 
required is therefore, in terms of arbitrary functions f and g, 
u(x, y) = f(x + iy) + g(x — iy). < 


It will be apparent from the last two examples that the nature of the appropriate 
linear combination of x and y depends upon whether B? > 4AC or B? < 4AC. 
This is exactly the same criterion as determines whether the PDE is hyperbolic 
or elliptic. Hence as a general result, hyperbolic and elliptic equations of the 
form (20.20), given the restriction that the constants A, B and C are real, have as 
solutions functions whose arguments have the form x+ay and x+ify respectively, 
where « and f themselves are real. 

The one case not covered by this result is that in which B? = 4AC, ie. a 
parabolic equation. In this case 2; and A, are not different and only one suitable 
combination of x and y results, namely 


u(x, y) = f(x — (B/2C)y). 


To find the second part of the general solution we try, in analogy with the 
corresponding situation for ordinary differential equations, a solution of the form 


u(x, y) = h(x, y)g(x — (B/2C)y). 
Substituting this into (20.20) and using A = B?/4C results in 
oh oh Oh 
A— + B— —}g=0. 
( Geo bey +c5) : 
Therefore we require h(x, y) to be any solution of the original PDE. There are 
several simple solutions of this equation, but as only one is required we take the 


simplest non-trivial one, h(x, y) = x, to give the general solution of the parabolic 
equation 


u(x, y) = f(x — (B/2C)y) + xg(x — (B/2C)y). (20.25) 
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We could, of course, have taken h(x, y) = y, but this only leads to a solution that 
is already contained in (20.25). 


Gee ae 


subject to the boundary conditions u(0,y) = 0 and u(x, 1) = x?. 


From our general result, functions of p = x + Ay will be solutions provided 
1421+? =0, 
ie. A = —1 and the equation is parabolic. The general solution is therefore 
u(x, y) = f(x —y) + xg(x— y). 
The boundary condition u(0, y) = 0 implies f(p) = 0, whilst u(x, 1) = x? yields 
xg(x — 1) =x’, 
which gives g(p) = p+ 1, Therefore the particular solution required is 


u(x, y) = x(p+1)=x(x-—y+1). < 


To reinforce the material discussed above we will now give alternative deriva- 
tions of the general solutions (20.22) and (20.25) by expressing the original PDE 
in terms of new variables before solving it. The actual solution will then become 
almost trivial; but, of course, it will be recognised that suitable new variables 
could hardly have been guessed if it were not for the work already done. This 
does not detract from the validity of the derivation to be described, only from 
the likelihood that it would be discovered by inspection. 

We start again with (20.20) and change to new variables 


€=x+Ay, n=xt+doy. 


With this change of variables, we have from the chain rule that 


(al a) (al 
an Ge ay 
é_,9,,4 
By OE Gy 


Using these and the fact that 
A+Bi,+C#?=0  fori=1,2, 


equation (20.20) becomes 


[2A + B(Ay + 42) + 2CAj Ad] 


oCOn 
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Then, providing the factor in brackets does not vanish, for which the required 
condition is easily shown to be B? + 4AC, we obtain 


au = 
alon 
which has the successive integrals 
ou : - 
3, Fla). wll.) = Fn) + (0). 
on 
This solution is just the same as (20.22), 
u(x, y) = f(x + Ary) + g(x + Ary). 
If the equation is parabolic (i.e. B? = 4AC), we instead use the new variables 
C=xt+dy, =x, 


and recalling that 2 = —(B/2C) we can reduce (20.20) to 


Two straightforward integrations give as the general solution 


u(l,n) =ng(l) + f(0), 


which in terms of x and y has exactly the form of (20.25), 


u(x, y) = xg(x + Ay) + f(x + Ay). 


Finally, as hinted at in subsection 20.3.2 with reference to first-order linear 
PDEs, some of the methods used to find particular integrals of linear ODEs 
can be suitably modified to find particular integrals of PDEs of higher order. In 
simple cases, however, an appropriate solution may often be found by inspection. 


> Find the general solution of 


Following our previous methods and results, the complementary function is 
u(x, y) = f(x + iy) + g(x — iy), 


and only a particular integral remains to be found. By inspection a particular integral of 
the equation is u(x, y) = x° + y’, and so the general solution can be written 


u(x,y) = f(x +iy) + g(x iy) +x? +. < 
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20.4 The wave equation 


We have already found that the general solution of the one-dimensional wave 
equation is 


u(x,t) = f(x —ct) + g(x + ct), (20.26) 


where f and g are arbitrary functions. However, the equation is of such general 
importance that further discussion will not be out of place. 

Let us imagine that u(x,t) = f(x—ct) represents the displacement of a string at 
time ¢ and position x. It is clear that all positions x and times t for which x—ct = 
constant will have the same instantaneous displacement. But x — ct = constant 
is exactly the relation between the time and position of an observer travelling 
with speed c along the positive x-direction. Consequently this moving observer 
sees a constant displacement of the string, whereas to a stationary observer, the 
initial profile u(x,0) moves with speed c along the x-axis as if it were a rigid 
system. Thus f(x — ct) represents a wave form of constant shape travelling along 
the positive x-axis with speed c, the actual form of the wave depending upon 
the function f. Similarly, the term g(x + ct) is a constant wave form travelling 
with speed c in the negative x-direction. The general solution (20.23) represents 
a superposition of these. 

If the functions f and g are the same then the complete solution (20.23) 
represents identical progressive waves going in opposite directions. This may 
result in a wave pattern whose profile does not progress, described as a standing 
wave. As a simple example, suppose both f(p) and g(p) have the form’ 


J (p) = g(p) = Acos(kp + €). 
Then (20.23) can be written as 
u(x, t) = A[cos(kx — kct + e) + cos(kx + kct + €)| 
= 2A cos(kct) cos(kx + €). 


The important thing to notice is that the shape of the wave pattern, given by the 
factor in x, is the same at all times but that its amplitude 24 cos(kct) depends 
upon time. At some points x that satisfy 


cos(kx + €) =0 


there is no displacement at any time; such points are called nodes. 

So far we have not imposed any boundary conditions on the solution (20.26). 
The problem of finding a solution to the wave equation that satisfies given bound- 
ary conditions is normally treated using the method of separation of variables 


S In the usual notation, k is the wave number (= 22/wavelength) and kc = a, the angular frequency 
of the wave. 
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discussed in the next chapter. Nevertheless, we now consider D’Alembert’s solution 
u(x,t) of the wave equation subject to initial conditions (boundary conditions) in 
the following general form: 


Ou(x, 0) 


initial displacement, u(x,0) = d(x); initial velocity, aE w(x). 


The functions f(x) and w(x) are given and describe the displacement and velocity 
of each part of the string at the (arbitrary) time t = 0. 

It is clear that what we need are the particular forms of the functions f and g 
in (20.26) that lead to the required values at t = 0. This means that 


(x) = u(x,0) = f(x—0) + g(x +0), ox 
vo) = M9) ep —0) + eel +0), Ce) 


where it should be noted that f’(x — 0) stands for df(p)/dp evaluated, after the 
differentiation, at p = x — c x 0; likewise for g’(x + 0). 

Looking on the above two left-hand sides as functions of p = x +ct, but 
everywhere evaluated at t = 0, we may integrate (20.28) between an arbitrary 
(and irrelevant) lower limit po and an indefinite upper limit p to obtain 


1 DP 
-/ w(q)dq+K =—f(p) + g(p), 


Po 
the constant of integration K depending on po. Comparing this equation with 
(20.27), with x replaced by p, we can establish the forms of the functions f and 
g as 


_ op) 1 f? K 

{QA 5, 2 w(g)dq— >, (20.29) 
_ op) , 1 f? K 

g(p) = a [ w(q)dq + > (20.30) 


Adding (20.29) with p = x —ct to (20.30) with p = x + ct gives as the solution to 
the original problem 


x+ct 


1 1 
u(x, t) = 5 [p(x — ct) + b(x + et)] + =| w(q) dq, (20.31) 


x—ct 
in which we notice that all dependence on po has disappeared. 

Each of the terms in (20.31) has a fairly straightforward physical interpretation. 
In each case the factor 1/2 represents the fact that only half a displacement profile 
that starts at any particular point on the string travels towards any other position 
x, the other half travelling away from it. The first term 5 P(X —ct) arises from 
the initial displacement at a distance ct to the left of x; this travels forward 
arriving at x at time ft. Similarly, the second contribution is due to the initial 
displacement at a distance ct to the right of x. The interpretation of the final 
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term is a little less obvious. It can be viewed as representing the accumulated 
transverse displacement at position x due to the passage past x of all parts of 
the initial motion whose effects can reach x within a time t, both backward and 
forward travelling. 

The extension to the three-dimensional wave equation of solutions of the type 
we have so far encountered presents no serious difficulty. In Cartesian coordinates 
the three-dimensional wave equation is 


Cu Ou ru 1 0?u 


Be ae oe eae G02?) 


In close analogy with the one-dimensional case we try solutions that are functions 
of linear combinations of all four variables, 


p=lx+my+nz + ut. 


It is clear that a solution u(x, y,z,t) = f(p) will be acceptable provided that 
2\ 724 
2 Foe Olt df (p) = 
( +m +n =) ap? 0. 


Thus, as in the one-dimensional case, f can be arbitrary provided that 


PH 4+? =p /c’. 
Using an obvious normalisation, we take « = +c and I, m, n as three numbers 
such that 
P+m+nr? =1. 


In other words (J,m,n) are the Cartesian components of a unit vector ni that 
points along the direction of propagation of the wave. The quantity p can be 
written in terms of vectors as the scalar expression p = fi-r ct, and the general 
solution of (20.32) is then 


u(x, y,Z,t) = u(r, t) = f(i-r—ct)+ g(-r + ct), (20.33) 


where fi is any unit vector. It would perhaps be more transparent to write fi 
explicitly as one of the arguments of u. 


20.5 The diffusion equation 


One important class of second-order PDEs, which we have not yet considered 
in detail, is that in which the second derivative with respect to one variable 
appears, but only the first derivative with respect to another (usually time). This 
is exemplified by the one-dimensional diffusion equation 

C?u(x,t) Au 


pola Sad ae ac 20.34 
0x2 ot’ ( ) 
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in which « is a constant with the dimensions length” x time !. The physical 
constants that go to make up x in a particular case depend upon the nature of 
the process (e.g. solute diffusion, heat flow, etc.) and the material being described. 

With (20.34) we cannot hope to repeat successfully the method of subsection 
20.3.3, since now u(x,t) is differentiated a different number of times on the two 
sides of the equation; any attempted solution in the form u(x,t) = f(p) with 
p = ax + bt will lead only to an equation in which the form of f cannot be 
cancelled out. Clearly we must try other methods. 

Solutions may be obtained by using the standard method of separation of 
variables discussed in the next chapter. Alternatively, a simple solution is also 
given if both sides of (20.34), as it stands, are separately set equal to a constant 
a (say), so that 


Cu oa ou 
= =a. 
Ox? Ke’ 


These equations have the general solutions 
u(x,t) = =x +xg(t)+h(t) and u(x,t) = at + m(x) 


respectively and may be made compatible with each other if g(t) is taken as 
constant, g(t) = g (where g could be zero), h(t) = at and m(x) = (a/2k)x? + gx. 
An acceptable solution is thus 


u(x, t) = =e + gx + at + constant. (20.35) 
K 


Let us now return to seeking solutions of equations by combining the inde- 
pendent variables in particular ways. Having seen that a linear combination of 
x and t will be of no value, we must search for other possible combinations. It 
has been noted already that « has the dimensions length? x time! and so the 
combination of variables 


will be dimensionless. Let us see if we can satisfy (20.34) with a solution of the 
form u(x,t) = f(y). Evaluating the necessary derivatives we have 

du _ df(n) on _ 2x df(n) 

Ox dyn Ox «Kt dy’ 


aL ae 


ax? xt. dy kt dy?’ 
du x? df(n) 
tt? dy 


Substituting these expressions into (20.34) we find that the new equation can be 
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written entirely in terms of 7, 


an HE HO 24 


+(2+)—— 


This is a straightforward ODE, which can be solved as follows. Writing f’(7) = 
df(n)/dn, etc., we have 


TN ath ce 
f(y) 2n 4 
= ee 


= fM=aR “exp (—) 


> rma al Pexp(— ) dp. 


If we now write this in terms of a slightly different variable 


yi? Pe 


(== 


2 ~ 2AxKt)!/2’ 


then df = gl? dn, and the solution to (20.34) is given by 


u(x,t) = f(y) =g(C) =B y exp(—v?) dy. (20.36) 


Here B is a constant and it should be noticed that x and t appear on the RHS 
only in the indefinite upper limit £, and then only in the combination xt~!/?. If 
is chosen as zero then u(x,t) is, to within a constant factor,’ the error function 
erf[x/2(«t)'/*], which is tabulated in many reference books. Only non-negative 
values of x and t are to be considered here, so that € > Co. 

Let us try to determine what kind of (say) temperature distribution and flow 
this represents. For definiteness we take () = 0. Firstly, since u(x,t) in (20.36) 
depends only upon the product xt~!/”, it is clear that all points x at times t such 
that xt~!/? has the same value have the same temperature. Put another way, at 
any specific time t the region having a particular temperature has moved along 
the positive x-axis a distance proportional to the square root of t. This is a typical 
diffusion process. 

Notice that, on the one hand, at t = 0 the variable € — oo and u becomes 
quite independent of x (except perhaps at x = 0); the solution then represents a 
uniform spatial temperature distribution. On the other hand, at x = 0 we have 
that u(x,t) is identically zero for all t. 


S Take B = 2n~'/? to give the usual error function normalised in such a way that erf(oo) = 1. See 
the Appendix. 
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> An infrared laser delivers a pulse of (heat) energy E to a point P on a large insulated 
sheet of thickness b, thermal conductivity k, specific heat s and density p. The sheet is 
initially at a uniform temperature. If u(r,t) is the excess temperature a time t later, at a 
point that is a distance r (> b) from P, then show that a suitable expression for u is 


u(r, t) = ~ exp (-=) (20.37) 


where « and B are constants. (Note that we use r instead of p to denote the radial coordinate 
in plane polars so as to avoid confusion with the density.) 

Further, (i) show that B = 2k/(sp); (ii) demonstrate that the excess heat energy in the 
sheet is independent of t, and hence evaluate a; and (iii) prove that the total heat flow past 
any circle of radius r is E. 


The equation to be solved is the heat diffusion equation 


Ou(r, t) 


kV? =s 
V-u(r, t) = sp ar 


Since we only require the solution for r >> b we can treat the problem as two-dimensional 
with obvious circular symmetry. Thus only the r-derivative term in the expression for V7u 


is non-zero, giving 
kod [0 0 
<= (3) = sp (20.38) 


where now u(r, t) = u(r, t). 


(i) Substituting the given expression (20.37) into (20.38) we obtain 


Qka [1 ; r\ — spa f(r i r 
(é-)aa)-#G)=C8) 


from which we find that (20.37) is a solution, provided B = 2k/(sp). 
(ii) The excess heat in the system at any time t is 


0 2p r 
bos | u(r, t)2ar dr = anbpss | 7 oxP (-=) dr 
= 2nbpsaf. 


The excess heat is therefore independent of t and so must be equal to the total heat 
input E, implying that 


ge a 
~ Inbpsp  4nbk’ 


(iii) The total heat flow past a circle of radius r is 


* Ou(r,t) 2 OE —r r 
—dnrbk | at a —2nrok f Tabi (=) exp (-s dt 


2 ive) 
E lexp ( mm)| E for all r. 
0 


As we would expect, all the heat energy E deposited by the laser will eventually flow past 
a circle of any given radius r. 4 
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20.6 Characteristics and the existence of solutions 


So far in this chapter we have discussed how to find general solutions to various 
types of first- and second-order linear PDE. Moreover, given a set of boundary 
conditions we have shown how to find the particular solution (or class of solutions) 
that satisfies them. For first-order equations, for example, we found that if the 
value of u(x, y) is specified along some curve in the xy-plane then the solution to 
the PDE is in general unique, but that if u(x, y) is specified at only a single point 
then the solution is not unique: there exists a class of particular solutions all of 
which satisfy the boundary condition. In this section and the next we make more 
rigorous the notion of the respective types of boundary condition that cause a 
PDE to have a unique solution, a class of solutions, or no solution at all. 


20.6.1 First-order equations 


Let us consider the general first-order PDE (20.9) but now write it as 


a 


; 
A(x, y) + B(x, y) = F(x, y,u). (20.39) 
y 


0 0 


Suppose we wish to solve this PDE subject to the boundary condition that 
u(x, y) = (s) is specified along some curve C in the xy-plane that is described 
parametrically by the equations x = x(s) and y = y(s), where s is the arc length 
along C. The variation of u along C is therefore given by 


du dudx | dudy _ dd 


Bone ee (20.40) 


We may then solve the two (inhomogeneous) simultaneous linear equations 
(20.39) and (20.40) for du/éx and du/éy, unless the determinant of the coefficients 
vanishes (see section 8.18), i.e. unless 


dx/ds_ dy/ds 


A B =0. 


At each point in the xy-plane this equation determines a set of curves called 
characteristic curves (or just characteristics), which thus satisfy 


B— —A—=0 
ds ds : 
or, multiplying through by ds/dx and dividing through by 4A, 
dy _ Bixy) 
eee . 20.41 
dx A(x, y) ( ) 


However, we have already met (20.41) in subsection 20.3.1 on first-order PDEs, 
where solutions of the form u(x, y) = f(p), where p is some combination of x and y, 
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were discussed. Comparing (20.41) with (20.12) we see that the characteristics are 
merely those curves along which p is constant. 

Since the partial derivatives 0u/0x and du/éy may be evaluated provided the 
boundary curve C does not lie along a characteristic, defining u(x,y) = ¢(s) 
along C is sufficient to specify the solution to the original problem (equation 
plus boundary conditions) near the curve C, in terms of a Taylor expansion 
about C. Therefore the characteristics can be considered as the curves along 
which information about the solution u(x, y) ‘propagates’. This is best understood 
by using an example. 


> Find the general solution of 


ou ou 
fae oa =0 (20.42) 


that takes the value 2y + 1 on the line x = 1 between y= 0 and y = 1. 


We solved this problem in subsection 20.3.1 for the case where u(x,y) takes the value 
2y +1 along the entire line x = 1. We found then that the general solution to the equation 
(ignoring boundary conditions) is of the form 


u(x, y) = f(p) = f(xy), 
for some arbitrary function f. Hence the characteristics of (20.42) are given by x*y =c 
where c is a constant; some of these curves are plotted in figure 20.2 for various values of 
c. Furthermore, we found that the particular solution for which u(1,y) = 2y + 1 for all y 
was given by 
u(x, y) = 2x°y +1. 

In the present case the value of x’y is fixed by the boundary conditions only between 
y =0 and y = 1. However, since the characteristics are curves along which x”y, and hence 
f(x’y), remains constant, the solution is determined everywhere along any characteristic 
that intersects the line segment denoting the boundary conditions. Thus u(x, y) = 2x?y +1 
is the particular solution that holds in the shaded region in figure 20.2 (corresponding to 
O0<c<1). 

Outside this region, however, the solution is not precisely specified, and any function of 
the form 


u(x, y) = 2x°y + 1+ g(x’y) 
will satisfy both the equation and the boundary condition, provided g(p) = 0 for 
O<p<l< 

In the above example the boundary curve was not itself a characteristic and 
furthermore it crossed each characteristic once only. For a general boundary curve 
C this may not be the case. Firstly, if C is itself a characteristic (or is just a single 
point) then information about the solution cannot ‘propagate’ away from C, and 
so the solution remains unspecified everywhere except on C. 

The second possibility is that C (although not a characteristic itself) crosses 
some characteristics more than once, as in figure 20.3. In this case specifying the 
value of u(x, y) along the curve PQ determines the solution along all the character- 
istics that intersect it. Therefore, also specifying u(x, y) along QR can overdetermine 
the problem solution and generally results in there being no solution. 
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Figure 20.2 The characteristics of equation (20.42). The shaded region shows 
where the solution to the equation is defined, given the imposed boundary 
condition at x = 1 between y = 0 and y = 1, shown as a bold vertical line. 


yi 


x 


Figure 20.3 A boundary curve C that crosses characteristics more than once. 


20.6.2 Second-order equations 


The concept of characteristics can be extended naturally to second- (and higher-) 
order equations. In this case let us write the general second-order linear PDE 
(20.19) as 


O7u Oru Cu ou Ou 
A(x, YG t BUS aay t Coon = F (su 2) m (20.43) 


Ox’ Oy 
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Figure 20.4 A boundary curve C and its tangent and unit normal at a given 
point. 


For second-order equations we might expect that relevant boundary conditions 
would involve specifying u, or some of its first derivatives, or both, along a 
suitable set of boundaries bordering or enclosing the region over which a solution 
is sought. Three common types of boundary condition occur and are associated 
with the names of Dirichlet, Neumann and Cauchy. They are as follows. 


(i) Dirichlet: The value of u is specified at each point of the boundary. 


(ii) Neumann: The value of du/én, the normal derivative of u, is specified at 
each point of the boundary. Note that du/dn = Vu- ii, where fi is the 
normal to the boundary at each point. 


(iti) Cauchy: Both u and du/0n are specified at each point of the boundary. 


Let us consider for the moment the solution of (20.43) subject to the Cauchy 
boundary conditions, i.e. u and du/én are specified along some boundary curve 
C in the xy-plane defined by the parametric equations x = x(s), y = y(s), s being 
the arc length along C (see figure 20.4). Let us suppose that along C we have 
u(x, y) = f(s) and du/én = y(s). At any point on C the vector dr = dxi+ dyj is 
a tangent to the curve and fids = dyi—dxj is a vector normal to the curve. Thus 
on C we have 


ou Vu dr _dudx  dudy _ ds) 
os ds @dxds édyds ds ’ 
ou ~  Oudy Oudx 
an eae ede 


These two equations may then be solved straightforwardly for the first partial 
derivatives 0u/0x and du/dy along C. Using the chain rule to write 
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we may differentiate the two first derivatives 0u/0x and du/dy along the boundary 
to obtain the pair of equations 


d fdu\ _ dx au dy u 
d (du\ _ dx ui dy @u 


We may now solve these two equations, together with the original PDE (20.43), 
for the second partial derivatives of u, except where the determinant of their 
coefficients equals zero, 


A BC 
dx dy 

ae as. 2 
9 & v 


Expanding out the determinant, 


dy\? dx\ (dy 
4(z) JC (f fit 


Multiplying through by (ds/dx)? we obtain 


2 
(a) 
ds 


(= ) 2 dy 
A B—+C=0, (20.44) 
dx dx 
which is the ODE for the curves in the xy-plane along which the second partial 
derivatives of u cannot be found. 

As for the first-order case, the curves satisfying (20.44) are called characteristics 
of the original PDE. These characteristics have tangents at each point given by 
(when A # 0) 


aos 
dy _ B+./B7—4A4C 04s) 


dx 2A : 


Clearly, when the original PDE is hyperbolic (B? > 4AC), equation (20.45) 
defines two families of real curves in the xy-plane; when the equation is parabolic 
(B? = 4AC) it defines one family of real curves; and when the equation is elliptic 
(B? < 4AC) it defines two families of complex curves. Furthermore, when A, 
B and C are constants, rather than functions of x and y, the equations of the 
characteristics will be of the form x + Ay = constant, which is reminiscent of the 
form of solution discussed in subsection 20.3.3. 
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ct h 
x — ct = constant 


x + ct = constant 


Figure 20.5 The characteristics for the one-dimensional wave equation. The 
shaded region indicates the region over which the solution is determined by 
specifying Cauchy boundary conditions at t = 0 on the line segment x = 0 to 
x=L. 


> Find the characteristics of the one-dimensional wave equation 


This is a hyperbolic equation with A = 1, B = 0 and C = —1/c?. Therefore from (20.44) 
the characteristics are given by 
2 
te es 
dt ; 


and so the characteristics are the straight lines x — ct = constant and x + ct = constant. < 


The characteristics of second-order PDEs can be considered as the curves along 
which partial information about the solution u(x, y) ‘propagates’. Consider a point 
in the space that has the independent variables as its coordinates; unless both 
of the two characteristics that pass through the point intersect the curve along 
which the boundary conditions are specified, the solution will not be determined 
at that point. In particular, if the equation is hyperbolic, so that we obtain two 
families of real characteristics in the xy-plane, then Cauchy boundary conditions 
propagate partial information concerning the solution along the characteristics, 
belonging to each family, that intersect the boundary curve C. The solution u 
is then specified in the region common to these two families of characteristics. 
For instance, the characteristics of the hyperbolic one-dimensional wave equation 
in the last example are shown in figure 20.5. By specifying Cauchy boundary 
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Equation type Boundary Conditions 


hyperbolic open Cauchy 
parabolic open Dirichlet or Neumann 
elliptic closed Dirichlet or Neumann 


Table 20.1 The appropriate boundary conditions for different types of partial 
differential equation. 


conditions u and du/dt on the line segment t = 0, x = 0 to L, the solution is 
specified in the shaded region. 

As in the case of first-order PDEs, however, problems can arise. For example, 
if for a hyperbolic equation the boundary curve intersects any characteristic 
more than once then Cauchy conditions along C can overdetermine the problem, 
resulting in there being no solution. In this case either the boundary curve C 
must be altered, or the boundary conditions on the offending parts of C must be 
relaxed to Dirichlet or Neumann conditions. 

The general considerations involved in deciding which boundary conditions are 
appropriate for a particular problem are complex, and we do not discuss them 
any further here’ We merely note that whether the various types of boundary 
condition are appropriate (in that they give a solution that is unique, sometimes 
to within a constant, and is well defined) depends upon the type of second-order 
equation under consideration and on whether the region of solution is bounded 
by a closed or an open curve (or a surface if there are more than two independent 
variables). Note that part of a closed boundary may be at infinity if conditions 
are imposed on u or du/On there. 

It may be shown that the appropriate boundary-condition and equation-type 
pairings are as given in table 20.1. 

For example, Laplace’s equation V7u = 0 is elliptic and thus requires either 
Dirichlet or Neumann boundary conditions on a closed boundary which, as we 
have already noted, may be at infinity if the behaviour of u is specified there 
(most often u or du/én — 0 at infinity). 


20.7 Uniqueness of solutions 


Although we have merely stated the appropriate boundary types and conditions 
for which, in the general case, a PDE has a unique, well-defined solution, some- 
times to within an additive constant, it is often important to be able to prove 
that a unique solution is obtained. 


S For a discussion the reader is referred, for example, to P. M. Morse and H. Feshbach, Methods of 
Theoretical Physics, Part I (New York: McGraw-Hill, 1953), chap. 6. 
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As an important example let us consider Poisson’s equation in three dimensions, 
V-u(r) = p(r), (20.46) 


with either Dirichlet or Neumann conditions on a closed boundary appropriate 
to such an elliptic equation; for brevity, in (20.46), we have absorbed any physical 
constants into p. We aim to show that, to within an unimportant constant, the 
solution of (20.46) is unique if either the potential u or its normal derivative 
0u/On is specified on all surfaces bounding a given region of space (including, if 
necessary, a hypothetical spherical surface of indefinitely large radius on which u 
or du/0n is prescribed to have an arbitrarily small value). Stated more formally 
this is as follows. 


Uniqueness theorem. If u is real and its first and second partial derivatives are 
continuous in a region V and on its boundary S, and V?u = p in V and either 
u =f or du/dn = g on S, where p, f and g are prescribed functions, then u is 
unique (at least to within an additive constant ). 


> Prove the uniqueness theorem for Poisson’s equation. 


Let us suppose on the contrary that two solutions u;(r) and up(r) both satisfy the conditions 
given above, and denote their difference by the function w = uw; — ux. We then have 


Vw = Vu, — Vu = p—p =0, 


so that w satisfies Laplace’s equation in V. Furthermore, since either u;) = f = uw or 
ou, /0n = g = Ouz/On on S, we must have either w = 0 or dOw/én = 0 on S. 

If we now use Green’s first theorem, (11.19), for the case where both scalar functions 
are taken as w we have 


i: [wV?w + (Vw): (Vw)] dV = | ue dS. 
V Ss on 


However, either condition, w = 0 or 0w/dén = 0, makes the RHS vanish whilst the first 
term on the LHS vanishes since V?w = 0 in V. Thus we are left with 


i |\VwP dV =0. 
Vv 


can never be negative, this can only be satisfied if 


Vw = 0, 


Since |Vw|? 


ie. if w, and hence u; — uo, is a constant in V. 

If Dirichlet conditions are given then u; = u2 on (some part of) S and hence u, = uy 
everywhere in V. For Neumann conditions, however, u,; and uz can differ throughout V 
by an arbitrary (but unimportant) constant. < 


The importance of this uniqueness theorem lies in the fact that if a solution to 
Poisson’s (or Laplace’s) equation that fits the given set of Dirichlet or Neumann 
conditions can be found by any means whatever, then that solution is the correct 
one, since only one exists. This result is the mathematical justification for the 
method of images, which is discussed more fully in the next chapter. 
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We also note that often the same general method, used in the above example 
for proving the uniqueness theorem for Poisson’s equation, can be employed to 
prove the uniqueness (or otherwise) of solutions to other equations and boundary 
conditions. 


20.8 Exercises 


20.1 Determine whether the following can be written as functions of p = x? + 2y only, 
and hence whether they are solutions of (20.8): 


(a) x2(x? — 4) + dy(x? — 2) +4? — 1); 
(b) x44 2x°y + y?s 
(c) [xt +4x2y + 4y? + 4] /[2x4 + x2(8y + 1) + 8y? + 2y]. 


20.2 Find partial differential equations satisfied by the following functions u(x, y) for 
all arbitrary functions f and all arbitrary constants a and b: 


(a) u(x,y) = f(x? — y’); 

(b) u(x,y) =(x-aP +(y —bY; 
(c) ux y) = y"f(y/x); 

(d) u(x, y) = f(x + ay) 


20.3 Solve the following partial differential equations for u(x, y) with the boundary 
conditions given: 


ou 


(a) xa +xy=u, u=2y on the line x = 1; 
Ox 
0 
(b) 14x =xu,  u(x,0) =x. 
oy 
20.4 Find the most general solutions u(x, y) of the following equations, consistent with 
the boundary conditions stated: 
0 0 
(a) yx =0, u(x,0) =1+sinx; 
Ox oy 
ee ge ae on ena 
ae age =y; 


j 0 
(c) sin x sin y— + cos xcos y— =0, u=cos2y onx+y=7/2; 
Ox oy 


0 


0 g 
(d) _ + 2x =0, u=2 on the parabola y = x?. 


20.5 Find solutions of 


lou lou | 
xOx yoy | 
for which (a) u(0, y) = y and (b) u(1,1) = 1. 
20.6 Find the most general solutions u(x, y) of the following equations consistent with 


the boundary conditions stated: 


0 6 
oe 3% u = x? on the line y = 0; 
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ou ou 
(b) va eT = 3x, u(1,0) = 2; 
20u | ,0U oo 3 3 iti 
(c) ag +x By =x y(x +y°), no boundary conditions. 
20.7 Solve 


. Ou i ou 
sin xa Oe, = cos x 
subject to (a) u(z/2, y) = 0 and (b) u(z/2, y) = y(y + 1). 
20.8 A function u(x, y) satisfies 
ou ou 
pate 
Ox oy 
and takes the value 3 on the line y = 4x. Evaluate u(2, 4). 
20.9 If u(x, y) satisfies 


= 10, 


07u eru Oru 


Ox Oxdy oy. 


and u = —x? and éu/éy = 0 for y = 0 and all x, find the value of u(0, 1). 
20.10 Consider the partial differential equation 
Ou Ou Ou 
— -—3— +2— = 0. * 
Ox? Oxdy * oy (*) 
(a) Find the function u(x, y) that satisfies (*) and the boundary condition u = 
odu/dy = 1 when y = 0 for all x. Evaluate u(0, 1). 
(b) In which region of the xy-plane would u be determined if the boundary 
condition were u = du/dy = 1 when y = 0 for all x > 0? 


20.11 In those cases in which it is possible to do so, evaluate u(2,2), where u(x, y) is 
the solution of 

ou 

ae 

Vox 


that satisfies the (separate) boundary conditions given below. 


0 
— x5 = xy(2y? — #7) 
oy 


(a) u(x, 1) = x? for all x. 
(b) u(x, 1) = x? for x > 0. 
(c) u(x, 1) =x? for0< x <3. 
(d) u(x,0) =x for x > 0. 
(e) u(x,0) = x for all x. 
(f) u(1,/10) = 5. 
(g) u(y70,1) =5. 
20.12 Solve 


Ou ‘ fu eu 
Ox? éxéy  éy2” 


subject to u = 2x + 1 and du/dy = 4 — 6x, both on the line y = 0. 
20.13 By changing the independent variables in the previous exercise to 


€=x+2y and n=x+3y, 
show that it must be possible to write 14(x? + Sxy + 6y’) in the form 
filx + 2y) + fo(x + 3y) — @? + y”), 


and determine the forms of f;(z) and f(z). 
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20.14 


20.15 
20.16 


20.17 


20.18 


20.19 


Solve 
Ou re Ou 
Oxdy oy? 
Find the most general solution of 67u/0x? + @7u/dy? = xy’. 
An infinitely long string on which waves travel at speed c has an initial displace- 
ment 


= x(2y + 3x). 


jaye con a<x<a, 
i bse 
It is released from rest at time t = 0, and its subsequent displacement is described 
by v(x, t). 

By expressing the initial displacement as one explicit function incorporating 

Heaviside step functions, find an expression for y(x,t) at a general time t > 0. In 
particular, determine the displacement as a function of time (a) at x = 0, (b) at 
x =a, and (c) atx =a/2. 
The non-relativistic Schrodinger equation (20.7) is similar to the diffusion equa- 
tion in having different orders of derivatives in its various terms; this precludes 
solutions that are arbitrary functions of particular linear combinations of vari- 
ables. However, since exponential functions do not change their forms under 
differentiation, solutions in the form of exponential functions of combinations of 
the variables may still be possible. 

Consider the Schrédinger equation for the case of a constant potential, i.e. for 
a free particle, and show that it has solutions of the form A exp(/x-+my +nz+At), 
where the only requirement is that 


Wo a ny ony 
mal! +m +7) = ihd. 


In particular, identify the equation and wavefunction obtained by taking 2 as 
—iE/h, and I,m and n as ip,/h,ipy/h and ip./h, respectively, where E is the 
energy and p the momentum of the particle; these identifications are essentially 
the content of the de Broglie and Einstein relationships. 
Like the Schrodinger equation of the previous exercise, the equation describing 
the transverse vibrations of a rod, 
Ou Pu 
a‘— —_ = 
éx*  6t? E 
has different orders of derivatives in its various terms. Show, however, that it 
has solutions of exponential form, u(x,t) = Aexp(Ax + iwt), provided that the 
relation a*/* = w” is satisfied. 

Use a linear combination of such allowed solutions, expressed as the sum of 
sinusoids and hyperbolic sinusoids of Ax, to describe the transverse vibrations of 
a rod of length L clamped at both ends. At a clamped point both u and du/dx 
must vanish; show that this implies that cos(AL) cosh(AL) = 1, thus determining 
the frequencies w at which the rod can vibrate. 

An incompressible fluid of density p and negligible viscosity flows with velocity v 
along a thin, straight, perfectly light and flexible tube, of cross-section A which is 
held under tension T. Assume that small transverse displacements u of the tube 
are governed by 

q2 


CEA ig Oi E T \ @u 
ot? Ox0t pA) dx? 


(a) Show that the general solution consists of a superposition of two waveforms 
travelling with different speeds. 
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20.20 


20.21 


20.22 


20.23 


20.24 


(b) The tube initially has a small transverse displacement u = acoskx and is 
suddenly released from rest. Find its subsequent motion. 


A sheet of material of thickness w, specific heat capacity c and thermal con- 
ductivity k is isolated in a vacuum, but its two sides are exposed to fluxes of 
radiant heat of strengths J; and Jy. Ignoring short-term transients, show that the 
temperature difference between its two surfaces is steady at (J2 —J1)w/2k, whilst 
their average temperature increases at a rate (J2 + J,)/cw. 

In an electrical cable of resistance R and capacitance C, each per unit length, 
voltage signals obey the equation 67V /dx? = RC@V /ét. This has solutions of the 
form given in (20.36) and also of the form V = Ax + D. 


(a) Find a combination of these that represents the situation after a steady 
voltage Vo is applied at x = 0 at time t = 0. 

(b) Obtain a solution describing the propagation of the voltage signal resulting 
from the application of the signal V = Vo for0 <t < T, V = 0 otherwise, 
to the end x = 0 of an infinite cable. 

(c) Show that for t >> T the maximum signal occurs at a value of x proportional 
to t? and has a magnitude proportional to ¢7!. 


The daily and annual variations of temperature at the surface of the earth may 
be represented by sine-wave oscillations, with equal amplitudes and periods of 
1 day and 365 days respectively. Assume that for (angular) frequency w the 
temperature at depth x in the earth is given by u(x,t) = A sin(wt + ux) exp(—Ax), 
where / and w are constants. 


(a) Use the diffusion equation to find the values of 2 and wu. 

(b) Find the ratio of the depths below the surface at which the two amplitudes 
have dropped to 1/20 of their surface values. 

(c) At what time of year is the soil coldest at the greater of these depths, 
assuming that the smoothed annual variation in temperature at the surface 
has a minimum on February Ist? 


Consider each of the following situations in a qualitative way and determine 
the equation type, the nature of the boundary curve and the type of boundary 
conditions involved: 


(a) aconducting bar given an initial temperature distribution and then thermally 
isolated; 

(b) two long conducting concentric cylinders, on each of which the voltage 
distribution is specified; 

(c) two long conducting concentric cylinders, on each of which the charge 
distribution is specified; 

(d) a semi-infinite string, the end of which is made to move in a prescribed way. 


This example gives a formal demonstration that the type of a second-order PDE 
(elliptic, parabolic or hyperbolic) cannot be changed by a new choice of independent 
variable. The algebra is somewhat lengthy, but straightforward. 
If a change of variable € = €(x,y), 4 = n(x, y) is made in (20.19), so that it 
reads 
Ou , Ou Ou Ou ou : pe 
ae +B BE0n +C an +D ae +E oF + Fu = R'(é,n), 


show that 


aém)]° 
B? —4d'C' =(B2 sac) s “ 
O(x, y) 
Hence deduce the conclusion stated above. 
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20.25 


20.1 
20.3 


20.5 
20.7 
20.9 


20.11 


20.13 


20.15 


20.17 


20.19 


20.21 


The Klein—Gordon equation (which is satisfied by the quantum-mechanical wave- 
function ®(r) of a relativistic spinless particle of non-zero mass m) is 


VO — mo = 0. 


Show that the solution for the scalar field ®(r) in any volume V bounded by 
a surface S is unique if either Dirichlet or Neumann boundary conditions are 
specified on S. 


20.9 Hints and answers 


(a) Yes, p* — 4p — 4; (b) no, (p — y)’; (c) yes, (p> + 4)/(2p? + p). 

Each equation is effectively an ordinary differential equation, but with a function 
of the non-integrated variable as the constant of integration; 

(a) u = xy(2—Inx); (b) u= x11 — e”) + xe’. 

(a) (2 — x)"; (b) 1+ f(y? — x2), where f(0) = 0. 

u=y-+f(y —In(sinx)); (a) u = In(sin x); (b) u = y + [y —In(sinx)]?. 

General solution is u(x,y) = f(x + y) + g(x + y/2). Show that 2p = —g'(p)/2, 
and hence g(p) = k — 2p’, whilst f(p) = p* — k, leading to u(x, y) = —x? + y?/2; 
u(0, 1) = 1/2. 

p=x° +2y*; ulx,y) = f(p) +x°y?/2. 


(a) u(x, y) = (x? + 2y? + x?y? — 2)/2; u(2,2) = 13. The line y = 1 cuts each 
characteristic in zero or two distinct points, but this causes no difficulty with 
the given boundary conditions. 

(b) As in (a). 

(c) The solution is defined over the space between the ellipses p = 2 and p = 11; 
(2, 2) lies on p = 12, and so u(2,2) is undetermined. 

(d) u(x, y) = (x? + 2y?)!? + x?y?/2; u(2,2) = 8 + /12. 

(e) The line y = 0 cuts each characteristic in two distinct points. No differentiable 
form of f(p) gives f(-ta) = +a respectively, and so there is no solution. 

(f) The solution is only specified on p = 21, and so u(2,2) is undetermined. 

(g) The solution is specified on p = 12, and so u(2,2) =5+ 3(4)(4) = 13. 


The equation becomes 67f /@€6y = —14, with solution f(é,1) = f(€)+g(n)—14én, 
which can be compared with the answer from the previous question; f(z) = 10z? 
and f(z) = 52?. 

u(x, y) = f(x + iy) + g(x — iy) + (1/12)x4(y? — (1/15)x?). In the last term, x and 
y may be interchanged. There are (infinitely) many other possibilities for the 
specific PI, e.g. [15x7y?(x? + y?) — (x® + y°) ]/360. 

E = p*/(2m), the relationship between energy and momentum for a non- 
relativistic particle; u(r, t) = Aexp[i(p-r — Et)/h], a plane wave of wave number 
k = p/h and angular frequency m = E/h travelling in the direction p/p. 

(a) c =v-+a where o? = T/pA; 

(b) u(x, t) = acos[k(x — vt)] cos(kat) — (va/a) sin[k(x — vt)] sin(kat). 


1 /2 
(a) Vo f Sp) ype” expl—¥") dy] : 
(b) consider the input as equivalent to Vo applied at t = 0 and continued and 
—Vo applied at t = T and continued; 
1 1/2 
QW prxteR/e-T) 
V(x,t) = a. exp (—v’) dv; 


Jt $(CR/1)'/2 


VoT exp(—4 
(c) For t >> T, maximum at x = [2t/(CR)]'/? with value — orP(= 3) 


(2n)'2¢ 
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20.23 


20.25 


(a) Parabolic, open, Dirichlet u(x,0) given, Neumann du/dx = 0 at x = +L/2 
for all t; 

(b) elliptic, closed, Dirichlet; 

(c) elliptic, closed, Neumann du/dn = a/e; 

(d) hyperbolic, open, Cauchy. 

Follow an argument similar to that in section 20.7 and argue that the additional 

term { m?|w|? dV must be zero, and hence that w = 0 everywhere. 
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Partial differential equations: 
separation of variables and other 
methods 


In the previous chapter we demonstrated the methods by which general solutions 
of some partial differential equations (PDEs) may be obtained in terms of 
arbitrary functions. In particular, solutions containing the independent variables 
in definite combinations were sought, thus reducing the effective number of them. 

In the present chapter we begin by taking the opposite approach, namely that 
of trying to keep the independent variables as separate as possible, using the 
method of separation of variables. We then consider integral transform methods 
by which one of the independent variables may be eliminated, at least from 
differential coefficients. Finally, we discuss the use of Green’s functions in solving 
inhomogeneous problems. 


21.1 Separation of variables: the general method 


Suppose we seek a solution u(x, y,z,t) to some PDE (expressed in Cartesian 
coordinates). Let us attempt to obtain one that has the product form’ 


u(x, y,z,t) = X(x)Y (y)Z(z)T (0). (21.1) 


A solution that has this form is said to be separable in x, y, z and t, and seeking 
solutions of this form is called the method of separation of variables. 
As simple examples we may observe that, of the functions 


(i) xyz? sin bt, (ii) xy + zt, (iii) (x* + y*)z cos wt, 


(i) is completely separable, (ii) is inseparable in that no single variable can be 
separated out from it and written as a multiplicative factor, whilst (iii) is separable 
in z and ¢ but not in x and y. 


S It should be noted that the conventional use here of upper-case (capital) letters to denote the 
functions of the corresponding lower-case variable is intended to enable an easy correspondence 
between a function and its argument to be made. 
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When seeking PDE solutions of the form (21.1), we are requiring not that 
there is no connection at all between the functions X, Y, Z and T (for example, 
certain parameters may appear in two or more of them), but only that X does 
not depend upon y, z, t, that Y does not depend on x, z, t, and so on. 

For a general PDE it is likely that a separable solution is impossible, but 
certainly some common and important equations do have useful solutions of 
this form, and we will illustrate the method of solution by studying the three- 
dimensional wave equation 


1 6?u(r) 
Vru(r) = = 
) ce Ot? 
We will work in Cartesian coordinates for the present and assume a solution 
of the form (21.1); the solutions in alternative coordinate systems, e.g. spherical 
or cylindrical polars, are considered in section 21.3. Expressed in Cartesian 
coordinates (21.2) takes the form 


(21.2) 


Cu Ou Pu 1 07u 


ae t ae eS eae? (21.3) 
substituting (21.1) gives 
OX yar exe ery xy terial yyz et 
dx? dy? dz? 2 dt?’ 
which can also be written as 
X"YZTH+XY"ZT+XYZ"T= SRO! (21.4) 


where in each case the primes refer to the ordinary derivative with respect to the 
independent variable upon which the function depends. This emphasises the fact 
that each of the functions X, Y, Z and T has only one independent variable and 
thus its only derivative is its total derivative. For the same reason, in each term 
in (21.4) three of the four functions are unaltered by the partial differentiation 
and behave exactly as constant multipliers. 
If we now divide (21.4) throughout by u = XY ZT we obtain 
x ye Z" 1 ae 
LOY ZS eT 
This form shows the particular characteristic that is the basis of the method of 
separation of variables, namely that of the four terms the first is a function of x 
only, the second of y only, the third of z only and the RHS a function of t only 
and yet there is an equation connecting them. This can only be so for all x, y, z 
and t if each of the terms does not in fact, despite appearances, depend upon the 
corresponding independent variable but is equal to a constant, the four constants 
being such that (21.5) is satisfied. 


(21.5) 
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Since there is only one equation to be satisfied and four constants involved, 
there is considerable freedom in the values they may take. For the purposes of 
our illustrative example let us make the choice of —/?, —m*, —n?, for the first 
three constants. The constant associated with c-?T"/T must then have the value 
—w =—-(? +m? +n”). 

Having recognised that each term of (21.5) is individually equal to a constant 
(or parameter), we can now replace (21.5) by four separate ordinary differential 
equations (ODEs): 

Xt P y” 4 Z" 4 1 i es 4 


= = =—)r. 21.6 


The important point to notice is not the simplicity of the equations (21.6) (the 
corresponding ones for a general PDE are usually far from simple) but that, by 
the device of assuming a separable solution, a partial differential equation (21.3), 
containing derivatives with respect to the four independent variables all in one 
equation, has been reduced to four separate ordinary differential equations (21.6). 
The ordinary equations are connected through four constant parameters that 
satisfy an algebraic relation. These constants are called separation constants. 

The general solutions of the equations (21.6) can be deduced straightforwardly 
and are 


X(x) = Aexp(ilx) + B exp(—ilx), 
Y(y) = Cexp(imy) + D exp(—imy), 


(21.7) 
Z(z) = Eexp(inz) + F exp(—inz), 
T(t) = Gexp(icut) + H exp(—icut), 
where A, B,...,H are constants, which may be determined if boundary condtions 


are imposed on the solution. Depending on the geometry of the problem and 
any boundary conditions, it is sometimes more appropriate to write the solutions 
(21.7) in the alternative form 

X(x) = A’ coslx + B’ sin 1x, 

Y(y) = C’cosmy + D'sinmy, 


(21.8) 
Z(z) = E'cosnz + F' sinnz, 
T(t) = G' cos(cut) + H’ sin(cut), 

for some different set of constants A’, B’,...,H’. Clearly the choice of how best 


to represent the solution depends on the problem being considered. 
As an example, suppose that we take as particular solutions the four functions 


X(x) =exp(ilx),  Y(y) = exp(imy), 
Z(z) = exp(inz), T(t) = exp(—icyt). 
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This gives a particular solution of the original PDE (21.3) 


u(x, y, Z,t) = exp(ilx) exp(imy) exp(inz) exp(—icyt) 
= exp[i(lx + my + nz — cut)], 


which is a special case of the solution (20.33) obtained in the previous chapter 
and represents a plane wave of unit amplitude propagating in a direction given 
by the vector with components /,m,n in a Cartesian coordinate system. In the 
conventional notation of wave theory, |, m and n are the components of the 
wave-number vector k, whose magnitude is given by k = 22/A, where 4 is the 
wavelength of the wave; cy is the angular frequency w of the wave. This gives 
the equation in the form 


u(x, y,z,t) = expli(kxx +kyy +k,z — ot)] 
= exp[i(k-r—ot)], 
and makes the exponent dimensionless. 


The method of separation of variables can be applied to many commonly 
occurring PDEs encountered in physical applications. 


>Use the method of separation of variables to obtain for the one-dimensional diffusion 
equation 
Ou du 
—S =F 21.9 
“Ox2 Ot’ ee 


a solution that tends to zero as t > 0 for all x. 


Here we have only two independent variables x and t and we therefore assume a solution 
of the form 


u(x,t) = X(x)T(t). 
Substituting this expression into (21.9) and dividing through by u = XT (and also by x) 


we obtain 
xX” _ T’ 
X KT’ 
Now, arguing exactly as above that the LHS is a function of x only and the RHS is a 
function of t only, we conclude that each side must equal a constant, which, anticipating 
the result and noting the imposed boundary condition, we will take as —/?. This gives us 
two ordinary equations, 
X"+7?X =0, (21.10) 
T'+2xT =0, (21.11) 
which have the solutions 
X(x) = Acos/x + B sin Ax, 
T(t) = C exp(—2xt). 
Combining these to give the assumed solution u = XT yields (absorbing the constant C 
into A and B) 


u(x,t) = (Acos Ax + B sin Ax) exp(—/?xt). (21.12) 
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In order to satisfy the boundary condition u — 0 as t > oo, 2x must be > 0. Since x 
is real and > 0, this implies that / is a real non-zero number and that the solution is 
sinusoidal in x and is not a disguised hyperbolic function; this was our reason for choosing 
the separation constant as —/?. < 


As a final example we consider Laplace’s equation in Cartesian coordinates; 
this may be treated in a similar manner. 


>Use the method of separation of variables to obtain a solution for the two-dimensional 
Laplace equation, 


Cu ou 
ae (21.13) 


If we assume a solution of the form u(x, y) = X(x)Y(y) then, following the above method, 
and taking the separation constant as 27, we find 


X"=2X, Y"=—-?Y. 
Taking /? as > 0, the general solution becomes 
u(x, y) = (Acoshdx + B sinh Ax)(C cos Ay + D sin Ay). (21.14) 


An alternative form, in which the exponentials are written explicitly, may be useful for 
other geometries or boundary conditions: 


u(x, y) = [Aexp Ax + Bexp(—Ax)](C cos Ay + D sin Ay), (21.15) 


with different constants A and B. 

If 2? < 0 then the roles of x and y interchange. The particular combination of sinusoidal 
and hyperbolic functions and the values of 4 allowed will be determined by the geometrical 
properties of any specific problem, together with any prescribed or necessary boundary 
conditions. < 


We note here that a particular case of the solution (21.14) links up with the 
‘combination’ result u(x, y) = f(x +iy) of the previous chapter (equations (20.24) 
and following), namely that if A = B and D =iC then the solution is the same 
as f(p) = AC exp Ap with p =x +iy. 


21.2 Superposition of separated solutions 


It will be noticed in the previous two examples that there is considerable freedom 
in the values of the separation constant /, the only essential requirement being 
that 2 has the same value in both parts of the solution, ie. the part depending 
on x and the part depending on y (or ft). This is a general feature for solutions 
in separated form, which, if the original PDE has n independent variables, will 
contain n — 1 separation constants. All that is required in general is that we 
associate the correct function of one independent variable with the appropriate 
functions of the others, the correct function being the one with the same values 
of the separation constants. 

If the original PDE is linear (as are the Laplace, Schrédinger, diffusion and 
wave equations) then mathematically acceptable solutions can be formed by 
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superposing solutions corresponding to different allowed values of the separation 
constants. To take a two-variable example: if 


Ua (XY) = X2, (x) Va, (y) 


is a solution of a linear PDE obtained by giving the separation constant the value 
A1, then the superposition 


u(x, y) = aX a,(x) Va, (y) + 2X3, (x) ¥2,(y) $2 = Dy aX i,(x)¥7,(y) 
(21.16) 


is also a solution for any constants aj, provided that the /; are the allowed values 
of the separation constant 2 given the imposed boundary conditions. Note that 
if the boundary conditions allow any of the separation constants to be zero then 
the form of the general solution is normally different and must be deduced by 
returning to the separated ordinary differential equations. We will encounter this 
behaviour in section 21.3. 

The value of the superposition approach is that a boundary condition, say that 
u(x, y) takes a particular form f(x) when y = 0, might be met by choosing the 
constants a; such that 


f(x) = $2 aX, (x) ¥;,(0). 


In general, this will be possible provided that the functions X;,(x) form a complete 
set — as do the sinusoidal functions of Fourier series or the spherical harmonics 
discussed in subsection 18.3. 


>A semi-infinite rectangular metal plate occupies the region0<x<aandO0<y<bin 


the xy-plane. The temperature at the far end of the plate and along its two long sides is 
fixed at 0°C. If the temperature of the plate at x = 0 is also fixed and is given by f(y), find 
the steady-state temperature distribution u(x,y) of the plate. Hence find the temperature 
distribution if f(y) = uo, where uo is a constant. 


The physical situation is illustrated in figure 21.1. With the notation we have used several 
times before, the two-dimensional heat diffusion equation satisfied by the temperature 


u(x, y, t) is 
Cu a Cu an 
K( a5 +55) =>. 
Ox? dy? ot 


with x = k/(sp). In this case, however, we are asked to find the steady-state temperature, 
which corresponds to du/ét = 0, and so we are led to consider the (two-dimensional) 
Laplace equation 


We saw that assuming a separable solution of the form u(x,y) = X(x)Y(y) led to 
solutions such as (21.14) or (21.15), or equivalent forms with x and y interchanged. In 
the current problem we have to satisfy the boundary conditions u(x,0) = 0 = u(x, b) and 
so a solution that is sinusoidal in y seems appropriate. Furthermore, since we require 
u(oo, y) = 0 it is best to write the x-dependence of the solution explicitly in terms of 
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y 
u=0 
b ae 
u= f(y) 4 u->0 
0 u=0 * 


Figure 21.1 A semi-infinite metal plate whose edges are kept at fixed tem- 
peratures. 


exponentials rather than of hyperbolic functions. We therefore write the separable solution 
in the form (21.15) as 


u(x, y) = [Aexp Ax + Bexp(—Ax)](C cos Ay + D sin Ay). 


Applying the boundary conditions, we see firstly that u(oo, y) = 0 implies A = 0 if we 
take 2 > 0. Secondly, since u(x,0) = 0 we may set C = 0, which, if we absorb the constant 
D into B, leaves us with 


u(x, y) = Bexp(—Ax) sin Ay. 


But, using the condition u(x,b) = 0, we require sin 2b = 0 and so 4 must be equal to nz/b, 
where n is any positive integer. 

Using the principle of superposition (21.16), the general solution satisfying the given 
boundary conditions can therefore be written 


u(x, y) = s B, exp(—n2x/b) sin(nzy/b), (21.17) 


n=1 


for some constants B,. Notice that in the sum in (21.17) we have omitted negative values of 
n since they would lead to exponential terms that diverge as x > oo. The n = 0 term is also 
omitted since it is identically zero. Using the remaining boundary condition u(0, y) = f(y) 
we see that the constants B, must satisfy 


f= > B, sin(nzy/b). (21.18) 


n=1 


This is clearly a Fourier sine series expansion of f(y) (see chapter 12). For (21.18) to 
hold, however, the continuation of f(y) outside the region 0 < y < b must be an odd 
periodic function with period 2b (see figure 21.2). We also see from figure 21.2 that if 
the original function f(y) does not equal zero at either of y = 0 and y = Db then its 
continuation has a discontinuity at the corresponding point(s); nevertheless, as discussed 
in chapter 12, the Fourier series will converge to the mid-points of these jumps and hence 
tend to zero in this case. If, however, the top and bottom edges of the plate were held not 
at 0°C but at some other non-zero temperature, then, in general, the final solution would 
possess discontinuities at the corners x = 0, y=0 and x =0, y=b. 

Bearing in mind these technicalities, the coefficients B, in (21.18) are given by 


a 2 [porsin (=) dy. (21.19) 
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Fy) 


Figure 21.2 The continuation of f(y) for a Fourier sine series. 


Therefore, if f(y) = uo (ie. the temperature of the side at x = 0 is constant along its 
length), (21.19) becomes 


b 


2uo b nny 
b nn cos ( b J. 

2uo h 4uo/nx for n odd, 
pares ta) ee 16 for n even. 


Therefore the required solution is 


u(x, y) = x 18 exp (-=) sin (=) .< 


n odd 


In the above example the boundary conditions meant that one term in each 
part of the separable solution could be immediately discarded, making the prob- 
lem much easier to solve. Sometimes, however, a little ingenuity is required in 
writing the separable solution in such a way that certain parts can be neglected 
immediately. 


> Suppose that the semi-infinite rectangular metal plate in the previous example is replaced 
by one that in the x-direction has finite length a. The temperature of the right-hand edge 
is fixed at 0°C and all other boundary conditions remain as before. Find the steady-state 
temperature in the plate. 


As in the previous example, the boundary conditions u(x, 0) = 0 = u(x, b) suggest a solution 
that is sinusoidal in y. In this case, however, we require u = 0 on x = a (rather than at 
infinity) and so a solution in which the x-dependence is written in terms of hyperbolic 
functions, such as (21.14), rather than exponentials is more appropriate. Moreover, since 
the constants in front of the hyperbolic functions are, at this stage, arbitrary, we may 
write the separable solution in the most convenient way that ensures that the condition 
u(a, y) = 0 is straightforwardly satisfied. We therefore write 


u(x, y) = [Acosh A(a — x) + B sinh A(a — x)](C cos Ay + D sin Ay). 
Now the condition u(a,y) = 0 is easily satisfied by setting A = 0. As before the 
conditions u(x,0) = 0 = u(x,b) imply C = 0 and 2 = nz/b for integer n. Superposing the 
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solutions for different n we then obtain 


(oe) 


u(x, y) = be B, sinh[na(a — x)/b] sin(nzy/b), (21.20) 


n=1 


for some constants B,. We have omitted negative values of n in the sum (21.20) since the 
relevant terms are already included in those obtained for positive n. Again the n = 0 term 
is identically zero. Using the final boundary condition u(0, y) = f(y) as above we find that 
the constants B, must satisfy 


f(y) = 95 Bn sinh(nna/b) sin(nzy/b), 
n=1 
and, remembering the caveats discussed in the previous example, the B,, are therefore given 


by 


2 Z . 
B= oe ip {(») sin(nry/b) dy. (21.21) 


For the case where f(y) = uo, following the working of the previous example gives 
(21.21) as 


4uo 
Bee See CRETIA| for n odd, B, =0 for n even. (21.22) 


The required solution is thus 


4uo a . 
u(x, y) De Ean (nna/B) sinh[nx(a — x)/b] sin (nzy/b) . 


We note that, as required, in the limit a > oo this solution tends to the solution of the 
previous example. < 


Often the principle of superposition can be used to write the solution to 
problems with more complicated boundary conditions as the sum of solutions to 
problems that each satisfy only some part of the boundary condition but when 
added togther satisfy all the conditions. 


> Find the steady-state temperature in the (finite) rectangular plate of the previous example, 
subject to the boundary conditions u(x, b) = 0, u(a, y) = 0 and u(0, y) = f(y) as before, but 
now, in addition, u(x,0) = g(x). 


Figure 21.3(c) shows the imposed boundary conditions for the metal plate. Although we 
could find a solution to this problem using the methods presented above, we can arrive at 
the answer almost immediately by using the principle of superposition and the result of 
the previous example. 

Let us suppose the required solution u(x, y) is made up of two parts: 


u(x, y) = v(x, y) + w(x, y), 


where v(x, y) is the solution satisfying the boundary conditions shown in figure 21.3(a), 
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yy y 
b iu b . 
10) 0 + o 0 
; OR a) ox 
(a) (b) 
y 
b 0 
= fy) 0 
g(x) a 
(c) 


Figure 21.3. Superposition of boundary conditions for a metal plate. 


whilst w(x, y) is the solution satisfying the boundary conditions in figure 21.3(b). It is clear 
that v(x, y) is simply given by the solution to the previous example, 


v(x, y) = ce B, sinh a sin (=) : 
n odd 


where B,, is given by (21.21). Moreover, by symmetry, w(x, y) must be of the same form as 
v(x, y) but with x and a interchanged with y and b, respectively, and with f(y) in (21.21) 
replaced by g(x). Therefore the required solution can be written down immediately without 
further calculation as 


u(x, y) = Le B, sinh od sin (=) + oe C,, sinh | sin (=) 7 


a 
n odd n odd 


the B, being given by (21.21) and the C, by 


2 ‘3 ; 
C, = saab g(x) sin(nax/a) dx. 


Clearly, this method may be extended to cases in which three or four sides of the plate 
have non-zero boundary conditions. < 


As a final example of the usefulness of the principle of superposition we now 
consider a problem that illustrates how to deal with inhomogeneous boundary 
conditions by a suitable change of variables. 
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>A bar of length L is initially at a temperature of 0°C. One end of the bar (x = 0) is held 
at 0°C and the other is supplied with heat at a constant rate per unit area of H. Find the 
temperature distribution within the bar after a time t. 


With our usual notation, the heat diffusion equation satisfied by the temperature u(x,t) is 


Ou du 
k— = FS, 
dx? at 
with « = k/(sp), where k is the thermal conductivity of the bar, s is its specific heat 
capacity and p is its density. 
The boundary conditions can be written as 


iO so:-- aonso, B® 
Ox k 


the last of which is inhomogeneous. In general, inhomogeneous boundary conditions can 
cause difficulties and it is usual to attempt a transformation of the problem into an 
equivalent homogeneous one. To this end, let us assume that the solution to our problem 
takes the form 


u(x, t) = v(x, t) + w(x), 


where the function w(x) is to be suitably determined. In terms of v and w the problem 


becomes 
ov dw dv 
(3 Ba =) = OF 
v(x, 0) + w(x) = 0, 
v(0, t) + w(0) = 0, 
Ov(L,t) dw(L) H 
Ox a dx k° 
There are several ways of choosing w(x) so as to make the new problem straightforward. 
Using some physical insight, however, it is clear that ultimately (at tf = oo), when all 


transients have died away, the end x = L will attain a temperature up such that kuo/L = H 
and there will be a constant temperature gradient u(x,00) = uox/L. We therefore choose 


Ax 


Since the second derivative of w(x) is zero, v satisfies the diffusion equation and the 
boundary conditions on v are now given by 


Ov(L, t) 
ox 


v(x,0) =, 00,2) =0, 


0. 
k , 


which are homogeneous in x. 
From (21.12) a separated solution for the one-dimensional diffusion equation is 
v(x, t) = (Acos Ax + B sin Ax) exp(—/?xt), 


corresponding to a separation constant —/?. If we restrict 2 to be real then all these 
solutions are transient ones decaying to zero as t > oo. These are just what is required to 
add to w(x) to give the correct solution as t > o. In order to satisfy v(0,t) = 0, however, 
we require A = 0. Furthermore, since 


a} > 
pe = Bexp(—/’xt)A cos Ax, 
ox 
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f(x) 


—HL/k+ : 


Figure 21.4 The appropriate continuation for a Fourier series containing 
only sine terms. 


in order to satisfy dv(L,t)/0x = 0 we require cosAL = 0, and so 2 is restricted to the 
values 


=> 
where n is an odd non-negative integer, ie. n = 1,3,5,. 
Thus, to satisfy the boundary condition v(x,0) = Hill, we must have 


s B, sin (=) _ =a, 


n odd 


in the range x = 0 to x = L. In this case we must be more careful about the continuation 
of the function —Hx/k, for which the Fourier sine series is required. We want a series that 
is odd in x (sine terms only) and continuous as x = 0 and x = L (no discontinuities, since 
the series must converge at the end-points). This leads to a continuation of the function 
as shown in figure 21.4, with a period of L’ = 4L. Following the discussion of section 12.3, 
since this continuation is odd about x = 0 and even about x = L’/4 = L it can indeed be 
expressed as a Fourier sine series containing only odd-numbered terms. 
The corresponding Fourier series coefficients are found to be 


—8HL (-1)"-)/2 


2 
kr? n 


for n odd, 


and thus the final formula for u(x,t) is 


ae 8HL (- a H/2 pa kren?t 
sc es ke a sin (SF) exp “AL? sp )? 


n odd 


giving the temperature for all positions 0 < x < L and for all times t > 0. < 


We note that in all the above examples the boundary conditions restricted the 
separation constant(s) to an infinite number of discrete values, usually integers. 
If, however, the boundary conditions allow the separation constant(s) 2 to take 
a Conese of values then the summation in (21.16) is replaced by an integral 
over 4. This is discussed further in connection with integral transform methods 
in section 21.4. 
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21.3 Separation of variables in polar coordinates 


So far we have considered the solution of PDEs only in Cartesian coordinates, 
but many systems in two and three dimensions are more naturally expressed 
in some form of polar coordinates, in which full advantage can be taken of 
any inherent symmetries. For example, the potential associated with an isolated 
point charge has a very simple expression, q/(4zeor), when polar coordinates are 
used, but involves all three coordinates and square roots when Cartesians are 
employed. For these reasons we now turn to the separation of variables in plane 
polar, cylindrical polar and spherical polar coordinates. 

Most of the PDEs we have considered so far have involved the operator V?, e.g. 
the wave equation, the diffusion equation, Schrédinger’s equation and Poisson’s 
equation (and of course Laplace’s equation). It is therefore appropriate that we 
recall the expressions for V? when expressed in polar coordinate systems. From 
chapter 10, in plane polars, cylindrical polars and spherical polars, respectively, 
we have 


1a/a 1 @ 
2 
Le ee 21.23 
- p 0p (os) +33 a eo 
1a/a 12 @ 
a 
a (6x) + ag t a (21.24) 
10/8 1 a a 1 @ 
2 2 e 
See te ee | a 21.25 
tr (: =) + sind 00 (sin =) © Dsl) Of? Oe 


Of course the first of these may be obtained from the second by taking z to be 
identically zero. 


21.3.1 Laplace’s equation in polar coordinates 


The simplest of the equations containing V? is Laplace’s equation, 
V-u(r) = 0. (21.26) 


Since it contains most of the essential features of the other more complicated 
equations, we will consider its solution first. 


Laplace’s equation in plane polars 


Suppose that we need to find a solution of (21.26) that has a prescribed behaviour 
on the circle p = a (e.g. if we are finding the shape taken up by a circular drumskin 
when its rim is slightly deformed from being planar). Then we may seek solutions 
of (21.26) that are separable in p and ¢ (measured from some arbitrary radius 
as ¢ = 0) and hope to accommodate the boundary condition by examining the 
solution for p = a. 
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Thus, writing u(p,) = P(p)®(d) and using the expression (21.23), Laplace’s 
equation (21.26) becomes 


® 0 ( OP P O® 
5, \P + 5g? = 2 
pop \' 6p p og 
Now, employing the same device as previously, that of dividing through by 
u = P® and multiplying through by p’, results in the separated equation 


po oP 10 
p t =0. 
Pop \ dp ® 0d? 
Following our earlier argument, since the first term on the RHS is a function of 
p only, whilst the second term depends only on ¢, we obtain the two ordinary 


equations 
CES (cash ee (21.27) 
Pdp \' dp 
12® 
oie —r, (21.28) 


where we have taken the separation constant to have the form n? for later 
convenience; for the present, n is a general (complex) number. 

Let us first consider the case in which n # 0. The second equation, (21.28), then 
has the general solution 


@(¢) = Aexp(ind) + B exp(—ing). (21.29) 
Equation (21.27), on the other hand, is the homogeneous equation 
pp” + pP’ = n-P = 0, 


which must be solved either by trying a power solution in p or by making the 
substitution p = expt as described in subsection 15.2.1 and so reducing it to an 
equation with constant coefficients. Carrying out this procedure we find 


P(p) = Cp" +Dp™. (21.30) 


Returning to the solution (21.29) of the azimuthal equation (21.28), we can 
see that if ®, and hence u, is to be single-valued and so not change when 
increases by 27 then n must be an integer. Mathematically, other values of n are 
permissible, but for the description of real physical situations it is clear that this 
limitation must be imposed. Having thus restricted the possible values of n in 
one part of the solution, the same limitations must be carried over into the radial 
part, (21.30). Thus we may write a particular solution of the two-dimensional 
Laplace equation as 


u(p,$) = (Acosnd + Bsinng)(Cp" + Dp~), 
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where A, B, C, D are arbitrary constants and n is any integer. 

We have not yet, however, considered the solution when n = 0. In this case, 
the solutions of the separated ordinary equations (21.28) and (21.27), respectively, 
are easily shown to be 


O() = AG +B, 
P(p)=Clnp +D. 


But, in order that u = P® is single-valued, we require A = 0, and so the solution 
for n = 0 is simply (absorbing B into C and D) 


u(p,d) = Clnp+D. 


Superposing the solutions for the different allowed values of n, we can write 
the general solution to Laplace’s equation in plane polars as 


u(p, &) = (Co ln p + Do) + ¥ “(An cosnd + By sinnd)(Crp” + Dip”), 
n=1 (21.31) 


where n can take only integer values. Negative values of n have been omitted 
from the sum since they are already included in the terms obtained for positive 
n. We note that, since Ing is singular at p = 0, whenever we solve Laplace’s 
equation in a region containing the origin, Co must be identically zero. 


>A circular drumskin has a supporting rim at p = a. If the rim is twisted so that it 
is displaced vertically by a small amount e(sing + 2sin2¢), where @ is the azimuthal 
angle with respect to a given radius, find the resulting displacement u(p, p) over the entire 
drumskin. 


The transverse displacement of a circular drumskin is usually described by the two- 
dimensional wave equation. In this case, however, there is no time dependence and so 
u(p,@) solves the two-dimensional Laplace equation, subject to the imposed boundary 
condition. 

Referring to (21.31), since we wish to find a solution that is finite everywhere inside 
p = 4, we require Co = 0 and D,, = 0 for all n > 0. Now the boundary condition at the 
rim requires 


u(a, pb) = Do + $3 C,a"(A, cosnd + B, sinnd) = e(sin d + 2sin2¢). 


n=1 


Firstly we see that we require Do = 0 and A, = 0 for all n. Furthermore, we must 
have C,B,a = €, C)B)a? = 2e and B, = 0 for n > 2. Hence the appropriate shape for the 
drumskin (valid over the whole skin, not just the rim) is 


_ ep, 2ep> . _epf. 20. 
upd) = Ling +E sin2g =F (sing +B sin2p .< 
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Laplace’s equation in cylindrical polars 


Passing to three dimensions, we now consider the solution of Laplace’s equation 
in cylindrical polar coordinates, 


10 ou 1@u @? 

pop (0) += ro ag? + ane =). (21.32) 
We note here that, even when considering a cylindrical physical system, if there 
is no dependence of the physical variables on z (ie. along the length of the 
cylinder) then the problem may be treated using two-dimensional plane polars, 
as discussed above. 

For the more general case, however, we proceed as previously by trying a 

solution of the form 


u(p, &,2) = P(p)®(b)Z(z), 
which, on substitution into (21.32) and division through by u = P®Z, gives 
Ldap). 1 do. 1¢Z _ 5 
Ip) Opde Zde 


The last term depends only on z, and the first and second (taken together) depend 
only on p and ¢. Taking the separation constant to be k*, we find 


1@Z 

os =k? 

Z dz? , 
1 d(aP 1 @o 
Paani (7, ea — + kk? = 0, 
Pp dp (> in) + Op ag > 


The first of these equations has the straightforward solution 
Z(z) = E exp(—kz) + F expkz. 


Multiplying the second equation through by p*, we obtain 
pd dP 1’o , ohh 
—— | p— k°p 

(> iP) oa 


in which the second term depends only on ® and the other terms depend only 
on p. Taking the second separation constant to be m?, we find 


1&0 
bie —m, (21.33) 
d ( aP 
Pip (0 =~) + (k?p* —m’)P = 0. (21.34) 


The equation in the azimuthal angle ¢ has the very familiar solution 


@(¢) = Ccosmd + Dsinm®¢. 
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As in the two-dimensional case, single-valuedness of u requires that m is an 
integer. However, in the particular case m = 0 the solution is 


(>) = Co +D. 


This form is appropriate to a solution with axial symmetry (C = 0) or one that is 
multivalued, but manageably so, such as the magnetic scalar potential associated 
with a current J (in which case C = I/(2z) and D is arbitrary). 

Finally, the p-equation (21.34) may be transformed into Bessel’s equation of 
order m by writing p = kp. This has the solution 


P(p) = AJn(kp) + BYm(kp). 


The properties of these functions were investigated in chapter 16 and will not 
be pursued here. We merely note that Y,,(kp) is singular at p = 0, and so, when 
seeking solutions to Laplace’s equation in cylindrical coordinates within some 
region containing the p = 0 axis, we require B = 0. 

The complete separated-variable solution in cylindrical polars of Laplace’s 
equation V7u = 0 is thus given by 


u(p, @, Z) = [AJn(kp) + BYn(kp)|[C cosmd + D sinm@][E exp(—kz) + F exp kz]. 
(21.35) 


Of course we may use the principle of superposition to build up more general 
solutions by adding together solutions of the form (21.35) for all allowed values 
of the separation constants k and m. 


>A semi-infinite solid cylinder of radius a has its curved surface held at 0°C and its base 


held at a temperature To. Find the steady-state temperature distribution in the cylinder. 


The physical situation is shown in figure 21.5. The steady-state temperature distribution 
u(p, @, z) must satisfy Laplace’s equation subject to the imposed boundary conditions. Let 
us take the cylinder to have its base in the z = 0 plane and to extend along the positive 
z-axis. From (21.35), in order that u is finite everywhere in the cylinder we immediately 
require B = 0 and F = 0. Furthermore, since the boundary conditions, and hence the 
temperature distribution, are axially symmetric, we require m = 0, and so the general 
solution must be a superposition of solutions of the form Jo(kp) exp(—kz) for all allowed 
values of the separation constant k. 

The boundary condition u(a,¢,z) = 0 restricts the allowed values of k, since we must 
have Jo(ka) = 0. The zeros of Bessel functions are given in most books of mathematical 
tables, and we find that, to two decimal places, 


Jo(x) =0 ~~ for x = 2.40, 5.52, 8.65, .... 


Writing the allowed values of k as k, for n = 1,2,3,... (so, for example, k; = 2.40/a), the 
required solution takes the form 


io 


u(p, ¢, z) = Sy Ay Jo(knp) exp(—k,,Z). 


n=1 
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a y 


oe u= To 


Figure 21.5 A uniform metal cylinder whose curved surface is kept at 0°C 
and whose base is held at a temperature Tp. 


x 


By imposing the remaining boundary condition u(p, 6,0) = To, the coefficients A, can be 
found in a similar way to Fourier coefficients but this time by exploiting the orthogonality 
of the Bessel functions, as discussed in chapter 16. From this boundary condition we 
require 


u(p, 6,0) = > AnJo(knp) = To- 


n=1 


If we multiply this expression by pJo(k,p) and integrate from p = 0 to p = a, and use the 
orthogonality of the Bessel functions Jo(k,), then the coefficients are given by (18.91) as 


aus i Jolknp)o dp. (21.36) 


An = sa 
05} (kn a) 0 


The integral on the RHS can be evaluated using the recurrence relation (18.92) of 
chapter 16, 


£ [eile] = 2Jo(2) 


dz 
which on setting z = k,p yields 


1d 
i, dp neh (knp)] = knpJolknp). 


Therefore the integral in (21.36) is given by 


a 


ig 1 1 
[f terredy = | osha), = eadbuay 
0 n 0 n 
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and the coefficients A, may be expressed as 


2To ad, (kna) 2To 
aI? (kya) k, k,aJ, (k,a) ; 


The steady-state temperature in the cylinder is then given by 


Ay 


u(p, b.2) = Se Tay on) exP(—kn2). 


We note that if, in the above example, the base of the cylinder were not kept at 
a uniform temperature Tp, but instead had some fixed temperature distribution 
T(p,@), then the solution of the problem would become more complicated. In 
such a case, the required temperature distribution u(p, d, z) is in general not axially 
symmetric, and so the separation constant m is not restricted to be zero but may 
take any integer value. The solution will then take the form 


u(p, , z) = by Se Iin(KnmP (Cam cos mo + Dim sin mo) exp(—knmZ), 


m=0 n=1 


where the separation constants ky, are such that Jin(knna) = 0, Le. Knna is the nth 
zero of the mth-order Bessel function. At the base of the cylinder we would then 
require 


u(p, ¢, 0) = 3 5 Jnl KnmP)(Cum cos mo + Drm sin mo) = T(p, p). 
m=0 n=1 (21.37) 


The coefficients Cy could be found by multiplying (21.37) by Jy(k-qp) cos q¢, 
integrating with respect to p and @ over the base of the cylinder and exploiting 
the orthogonality of the Bessel functions and of the trigonometric functions. The 
Dym could be found in a similar way by multiplying (21.37) by Jg(k,qp) sing®¢. 


Laplace’s equation in spherical polars 


We now come to an equation that is very widely applicable in physical science, 
namely V7u = 0 in spherical polar coordinates: 


10 (»éu 1 @/f. ,éu 1 @u 
[PS —— — G— —- = = 0. 21.38 
2 OF ¢ 4) + sind 00 (si a) + sin 0 Oe oe 
Our method of procedure will be as before; we try a solution of the form 
u(r, 0,b) = R(r)O(O)®(¢). 
Substituting this in (21.38), dividing through by u = RO® and multiplying by r’, 
we obtain 


ld /.dR\, 1 df. d@O\, 1 @o_ 
Rdr (« =) ' @sin 6d dO (sino) " sin? 0 de 0. (21.39) 
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The first term depends only on r and the second and third terms (taken together) 
depend only on 6 and ¢. Thus (21.39) is equivalent to the two equations 


1d (odR\_, 
Rdr ¢ =) ec (21.40) 
Osin0 dd (sino 7B) Vpaode (21.41) 


Equation (21.40) is a homogeneous equation, 


CR dR 

2 

~—— + 2r— —AIR=0 
dr? ne dr ; 


which can be reduced, by the substitution r = expt (and writing R(r) = S(t)), to 


@S dS 
ar t qs =0 


This has the straightforward solution 
S(t) = AexpA,t + Bexp Agt, 


and so the solution to the radial equation is 


R(r) = Ar” + Br®, 


where 4; + 22 = —1 and 4,4. = —/. We can thus take 4; and 2 as given by 7 
and —(/+ 1); 4 then has the form 7(¢+ 1). (It should be noted that at this stage 
nothing has been either assumed or proved about whether / is an integer.) 

Hence we have obtained some information about the first factor in the 
separated-variable solution, which will now have the form 


u(r,0,@) = [Ar’ + Br] ©(0)®(¢), (21.42) 


where © and © must satisfy (21.41) with 2 = 7(¢ + 1). 
The next step is to take (21.41) further. Multiplying through by sin? @ and 

substituting for 4, it too takes a separated form: 

10 = 


Sie 0. (21.43) 


sinOdd /. dO 7) 
we (sino) + ¢(¢+1)sin° 0) + 


Taking the separation constant as m?, the equation in the azimuthal angle ¢ 
has the same solution as in cylindrical polars, namely 


@(¢) = Ccosmd + D sinm®¢. 


As before, single-valuedness of u requires that m is an integer; for m = 0 we again 
have O(f) = Ch+D. 
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Having settled the form of ®(¢), we are left only with the equation satisfied by 
©(0), which is 


sin@ d * dO - 29,2 
ow (sin 0s) + ¢(¢ + 1)sin° 0 =m’. (21.44) 
A change of independent variable from 0 to 4 = cos@ will reduce this to a 
form for which solutions are known, and of which some study has been made in 
chapter 16. Putting 


7 du, d niyo 4 
Lh =cos8, a9 = 8108, 7 ) ae 


the equation for M(u) = ©(0) reads 


d 


>,d4M ; = 
i a | lac tI) 73] M=0 (21.45) 


This equation is the associated Legendre equation, which was mentioned in sub- 
section 18.2 in the context of Sturm—Liouville equations. 

We recall that for the case m = 0, (21.45) reduces to Legendre’s equation, which 
was studied at length in chapter 16, and has the solution 


M(u) = EP-(u) + FQ/(w). (21.46) 


We have not solved (21.45) explicitly for general m, but the solutions were given 
in subsection 18.2 and are the associated Legendre functions P/"(u) and Q?'(11), 
where 


|m| 


d 
Pn) = (1 — py Ps (21.47) 


and similarly for Q?(u). We then have 
M(u) = EP?(u) + FO? (u); (21.48) 


here m must be an integer, 0 < |m| < 7. We note that if we require solutions to 
Laplace’s equation that are finite when w = cos? = +1 (ie. on the polar axis 
where 0 = 0,7), then we must have F = 0 in (21.46) and (21.48) since Q?'(u) 
diverges at wp = +1. 

It will be remembered that one of the important conditions for obtaining 
finite polynomial solutions of Legendre’s equation is that 7 is an integer > 0. 
This condition therefore applies also to the solutions (21.46) and (21.48) and is 
reflected back into the radial part of the general solution given in (21.42). 

Now that the solutions of each of the three ordinary differential equations 
governing R, © and © have been obtained, we may assemble a complete separated- 
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variable solution of Laplace’s equation in spherical polars. It is 


u(r, 0,@) = (Ar? + Br )(C cosm@ + D sinm¢)[EP2"(cos 0) + FO" (cos 0)], 
(21.49) 


where the three bracketted factors are connected only through the integer pa- 
rameters 7 and m, 0 < |m| < /. As before, a general solution may be obtained 
by superposing solutions of this form for the allowed values of the separation 
constants 7 and m. As mentioned above, if the solution is required to be finite on 
the polar axis then F = 0 for all 7 and m. 


> An uncharged conducting sphere of radius a is placed at the origin in an initially uniform 
electrostatic field E. Show that it behaves as an electric dipole. 


The uniform field, taken in the direction of the polar axis, has an electrostatic potential 
u = —Ez = —Ercos0, 


where u is arbitrarily taken as zero at z = 0. This satisfies Laplace’s equation Vu = 0, as 
must the potential v when the sphere is present; for large r the asymptotic form of v must 
still be —Er cos 0. 

Since the problem is clearly axially symmetric, we have immediately that m = 0, and 
since we require v to be finite on the polar axis we must have F = 0 in (21.49). Therefore 
the solution must be of the form 


oa) 


v(r, 0, ) = SoAer’ + Byr))P,(cos 8). 


/=0 


Now the cos 6-dependence of v for large r indicates that the (0, p)-dependence of v(r, 0, d) 
is given by P?(cos@) = cos@. Thus the r-dependence of v must also correspond to an 
¢ =1 solution, and the most general such solution (outside the sphere, i.e. for r > a) is 


v(r, 0, @) = (Ayr + Byr*)P; (cos 8). 


The asymptotic form of v for large r immediately gives A; = —E and so yields the solution 


v(r, 0) = (-e + =) Sosa: 


Since the sphere is conducting, it is an equipotential region and so v must not depend on 
0 for r =a. This can only be the case if B,/a? = Ea, thus fixing B,. The final solution is 
therefore 


r 


a 
u(r, 0, ) = —Er (1 - =) cos 0. 


Since a dipole of moment p gives rise to a potential p/(4meor7), this result shows that the 
sphere behaves as a dipole of moment 4ze9a°E, because of the charge distribution induced 
on its surface; see figure 21.6. < 


Often the boundary conditions are not so easily met, and it is necessary to 
use the mutual orthogonality of the associated Legendre functions (and the 
trigonometric functions) to obtain the coefficients in the general solution. 
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Figure 21.6 Induced charge and field lines associated with a conducting 
sphere placed in an initially uniform electrostatic field. 


>A hollow split conducting sphere of radius a is placed at the origin. If one half of its 
surface is charged to a potential vo and the other half is kept at zero potential, find the 
potential v inside and outside the sphere. 


Let us choose the top hemisphere to be charged to vo and the bottom hemisphere to be 
at zero potential, with the plane in which the two hemispheres meet perpendicular to the 
polar axis; this is shown in figure 21.7. The boundary condition then becomes 


; _ f vu for0<@0<2/2 (0<cosé <1), 
H(a,8,6) = { 0 forn/2<0<nx (-1<cos <0). (21.50) 


The problem is clearly axially symmetric and so we may set m = 0. Also, we require the 
solution to be finite on the polar axis and so it cannot contain Q;(cos 0). Therefore the 
general form of the solution to (21.38) is 


v(r,0,) = Sider! + Br) P/(cos 0). (21.51) 


¢=0 


Inside the sphere (for r < a) we require the solution to be finite at the origin and so 
B, = 0 for all 7 in (21.51). Imposing the boundary condition at r = a we must then have 


v(a,0,¢) = )> Ara’ P/(cos 0), 
¢=0 


where v(a, 0, #) is also given by (21.50). Exploiting the mutual orthogonality of the Legendre 
polynomials, the coefficients in the Legendre polynomial expansion are given by (18.14) 
as (writing 4 = cos 0) 


2~+1 f 
Acad = E* _ v(a,0, 6)PApidu 
Z| 


1 1 
= 5 wf Pe(wdu, 
0 
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Figure 21.7 A hollow split conducting sphere with its top half charged to a 
potential vo and its bottom half at zero potential. 


where in the last line we have used (21.50). The integrals of the Legendre polynomials are 
easily evaluated (see exercise 17.3) and we find 


= vo 3v0 = oe Tvo 
Ao = >> =~ A,=0, A3= Teas’ , 


so that the required solution inside the sphere is 


vo 
2 


v(r, 0, b) 


3 
ald P;(cos 0) Us P3(cos0) +--- 
a 8a3 


[+5 


Outside the sphere (for r > a) we require the solution to be bounded as r tends to 
infinity and so in (21.51) we must have A; = 0 for all 7. In this case, by imposing the 
boundary condition at r = a we require 


v(a,0,¢) = S> Bea“ P,(cos 0), 


=0 
where v(a,0,) is given by (21.50). Following the above argument the coefficients in the 
expansion are given by 


= 2w+1 : 
Beat) = EFT / Po(wdy, 
0 


so that the required solution outside the sphere is 


3 


Ta’ 
a7 P3(cos0@) +--+}. <4 


vod 


2r 


v(r, 0, ) res =“ Pi(cos0) 
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In the above example, on the equator of the sphere (i.e. at r = a and 0 = 1/2) 
the potential is given by 


v(a, 1/2, ¢) = vo/2, 


ie. mid-way between the potentials of the top and bottom hemispheres. This is 
so because a Legendre polynomial expansion of a function behaves in the same 
way as a Fourier series expansion, in that it converges to the average of the two 
values at any discontinuities present in the original function. 

If the potential on the surface of the sphere had been given as a function of 0 
and @, then we would have had to consider a double series summed over ¢ and 
m (for —¢ < m < “), since, in general, the solution would not have been axially 
symmetric. 

Finally, we note in general that, when obtaining solutions of Laplace’s equation 
in spherical polar coordinates, one finds that, for solutions that are finite on the 
polar axis, the angular part of the solution is given by 


©(0)®(d) = P7"(cos 0)(C cosmd + D sinm®), 


where / and m are integers with —¢ < m < /. This general form is sufficiently 
common that particular functions of 6 and ¢ called spherical harmonics are 
defined and tabulated (see section 18.3). 


21.3.2 Other equations in polar coordinates 


The development of the solutions of V?u = 0 carried out in the previous subsection 
can be employed to solve other equations in which the V* operator appears. Since 
we have discussed the general method in some depth already, only an outline of 
the solutions will be given here. 

Let us first consider the wave equation 
2u 


oa (21.52) 


ind) 


1 
Dims 
VERt 


D 


and look for a separated solution of the form u = F(r)T(t), so that initially we 
are separating only the spatial and time dependences. Substituting this form into 
(21.52) and taking the separation constant as k* we obtain 


2 
VEF+KRF=0, —4+k°?T =0. (21.53) 

The second equation has the simple solution 
T(t) = Aexp(iat) + B exp(—iot), (21.54) 


where w = kc; this may also be expressed in terms of sines and cosines, of course. 
The first equation in (21.53) is referred to as Helmholtz’s equation; we discuss it 
below. 
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We may treat the diffusion equation 


ou 
«Vu = Tae 
ot 


in a similar way. Separating the spatial and time dependences by assuming a 
solution of the form u = F(r)T(t), and taking the separation constant as k?, we 
find 


dT 
VF+kKF =0, ae +kKT =0. 


Just as in the case of the wave equation, the spatial part of the solution satisfies 
Helmholtz’s equation. It only remains to consider the time dependence, which 
has the simple solution 


T(t) = A exp(—k’xt). 
Helmholtz’s equation is clearly of central importance in the solutions of the 
wave and diffusion equations. It can be solved in polar coordinates in much the 
same way as Laplace’s equation, and indeed reduces to Laplace’s equation when 


k =0. Therefore, we will merely sketch the method of its solution in each of the 
three polar coordinate systems. 


Helmholtz’s equation in plane polars 
In two-dimensional plane polar coordinates, Helmholtz’s equation takes the form 
a 2 
mea cally meats agian ce 
pop dp] p? ¢? 
If we try a separated solution of the form F(r) = P(p)®(¢), and take the 


separation constant as m’, we find 
Oo , 
oes =0 
Ip +m = 0, 
’P  1dP ( me 
Sp ep e-5)p=o, 
dp? p dp p 


As for Laplace’s equation, the angular part has the familiar solution (if m # 0) 
O(¢) = Acosm¢d + B sinmd, 


or an equivalent form in terms of complex exponentials. The radial equation 
differs from that found in the solution of Laplace’s equation, but by making the 
substitution = kp it is easily transformed into Bessel’s equation of order m 
(discussed in chapter 16), and has the solution 


where Y,, is a Bessel function of the second kind, which is infinite at the origin 
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and is not to be confused with a spherical harmonic (these are written with a 
superscript as well as a subscript). 
Putting the two parts of the solution together we have 


F(p,$) = [Acosmd + Bsinmg][CJn(kp) + DY nlkp)|. (21.55) 


Clearly, for solutions of Helmholtz’s equation that are required to be finite at the 
origin, we must set D = 0. 


> Find the four lowest frequency modes of oscillation of a circular drumskin of radius a 
whose circumference is held fixed in a plane. 


The transverse displacement u(r,t) of the drumskin satisfies the two-dimensional wave 
equation 


1 u 
2 Ot?’ 
with c? = T/o, where T is the tension of the drumskin and o is its mass per unit area. 


From (21.54) and (21.55) a separated solution of this equation, in plane polar coordinates, 
that is finite at the origin is 


u(p, d, t) = Jn(kp)(A cos md + B sin m@) exp(tiot), 


Vu= 


where w = kc. Since we require the solution to be single-valued we must have m as an 
integer. Furthermore, if the drumskin is clamped at its outer edge p = a then we also 
require u(a, b,t) = 0. Thus we need 


Im(ka) = 0, 


which in turn restricts the allowed values of k. The zeros of Bessel functions can be 
obtained from most books of tables; the first few are 


Jo(x) =0 for x 2.40, 5.52, 8.65,..., 
Ji(x)=0 for x = 3.83, 7.02, 10.17,..., 
Jo(x) =0 for x = 5.14, 8.42, 11.62.... 
The smallest value of x for which any of the Bessel functions is zero is x = 2.40, which 
occurs for Jo(x). Thus the lowest-frequency mode has k = 2.40/a and angular frequency 
@ = 2.40c/a. Since m = 0 for this mode, the shape of the drumskin is 


uc Jo (2.404) : 


this is illustrated in figure 21.8. 
Continuing in the same way, the next three modes are given by 


= 3,835 p PY sing: 
o= 3835, wed, (3.83 *) cosd, J (3.83 ) sing: 
25142 id ®) sina: 
o = 5.14-, uc Jo (5.142) cos 24, Jy (5.142) sin2¢; 
@ = 5.525, woe Jo(5.524). 
a a 
These modes are also shown in figure 21.8. We note that the second and third frequencies 


have two corresponding modes of oscillation; these frequencies are therefore two-fold 
degenerate. < 
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@ =2.40c/a 


@ =5.14c/a w = 5.52c/a 


Figure 21.8 The modes of oscillation with the four lowest frequencies for a 
circular drumskin of radius a. The dashed lines indicate the nodes, where the 
displacement of the drumskin is always zero. 


Helmholtz’s equation in cylindrical polars 


Generalising the above method to three-dimensional cylindrical polars is straight- 
forward, and following a similar procedure to that used for Laplace’s equation 
we find the separated solution of Helmholtz’s equation takes the form 


F(p,,2) = [Ayn (Vie =a? p) + BYn (Vie = a2 r)| 


x (Ccosm¢d + D sinm@)[E exp(iaz) + F exp(—iaz)], 


where « and m are separation constants. We note that the angular part of the 
solution is the same as for Laplace’s equation in cylindrical polars. 


Helmholtz’s equation in spherical polars 


In spherical polars, we find again that the angular parts of the solution ©(0)®(¢) 
are identical to those of Laplace’s equation in this coordinate system, i.e. they are 
the spherical harmonics Y/"(0,), and so we shall not discuss them further. 

The radial equation in this case is given by 


rR" + 2rR' + [k?r? —¢(¢ + DIR = 0, (21.56) 


which has an additional term k?7?R compared with the radial equation for the 
Laplace solution. The equation (21.56) looks very much like Bessel’s equation. 
In fact, by writing R(r) = r—'/?S(r) and making the change of variable p = kr, 
it can be reduced to Bessel’s equation of order 7 + 5 which has as its solutions 
S(w) = Jrztj2(u) and Y,+1/2(u) (see section 18.6). The separated solution to 
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Helmholtz’s equation in spherical polars is thus 


F(r,0,¢) =r! [AJy412(kr) + BY;41;2(kr)](C cosm¢ + D sinm®@) 
x[EP?"(cos 0) + FQ7(cos 0)]. (21.57) 


For solutions that are finite at the origin we require B = 0, and for solutions 
that are finite on the polar axis we require F = 0. It is worth mentioning that 
the solutions proportional to rT (kr) and r—!/2 Y/41/2(kr), when suitably 
normalised, are called spherical Bessel functions of the first and second kind, 
respectively, and are denoted by j/(kr) and n;() (see section 18.6). 

As mentioned at the beginning of this subsection, the separated solution of 
the wave equation in spherical polars is the product of a time-dependent part 
(21.54) and a spatial part (21.57). It will be noticed that, although this solution 
corresponds to a solution of definite frequency w = kc, the zeros of the radial 
function j;(kr) are not equally spaced in r, except for the case 7 = 0 involving 
jo(kr), and so there is no precise wavelength associated with the solution. 

To conclude this subsection, let us mention briefly the Schrodinger equation 
for the electron in a hydrogen atom, the nucleus of which is taken at the origin 
and is assumed massive compared with the electron. Under these circumstances 
the Schrodinger equation is 


rh 2 eu _ nou 
2m 4neo Tr : Ot 
For a ‘stationary-state’ solution, for which the energy is a constant E and the time- 
dependent factor T in uw is given by T(t) = A exp(—iEt/h), the above equation is 
similar to, but not quite the same as, the Helmholtz equation’ However, as with 
the wave equation, the angular parts of the solution are identical to those for 
Laplace’s equation and are expressed in terms of spherical harmonics. 

The important point to note is that for any equation involving V, provided 0 
and ¢ do not appear in the equation other than as part of V, a separated-variable 
solution in spherical polars will always lead to spherical harmonic solutions. This 
is the case for the Schrodinger equation describing an atomic electron in a central 
potential V(r). 


21.3.3 Solution by expansion 


It is sometimes possible to use the uniqueness theorem discussed in the previous 
chapter, together with the results of the last few subsections, in which Laplace’s 
equation (and other equations) were considered in polar coordinates, to obtain 
solutions of such equations appropriate to particular physical situations. 


S For the solution by series of the r-equation in this case the reader may consult, for example, L. 
Schiff, Quantum Mechanics (New York: McGraw-Hill, 1955), p. 82. 
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a y 


Figure 21.9 The polar axis Oz is taken as normal to the plane of the ring of 
matter and passing through its centre. 


We will illustrate the method for Laplace’s equation in spherical polars and first 
assume that the required solution of V2u = 0 can be written as a superposition 
in the normal way: 


va) t 
u(r, 0,@) = oa x: (Ar? + Br“) P"(cos 0)(C cosmd + D sinm@). 
pee (21.58) 


Here, all the constants A, B,C, D may depend upon / and m, and we have 
assumed that the required solution is finite on the polar axis. As usual, boundary 
conditions of a physical nature will then fix or eliminate some of the constants; 
for example, u finite at the origin implies all B = 0, or axial symmetry implies 
that only m = 0 terms are present. 

The essence of the method is then to find the remaining constants by determin- 
ing u at values of r, 0, @ for which it can be evaluated by other means, e.g. by direct 
calculation on an axis of symmetry. Once the remaining constants have been fixed 
by these special considerations to have particular values, the uniqueness theorem 
can be invoked to establish that they must have these values in general. 


> Calculate the gravitational potential at a general point in space due to a uniform ring of| 
matter of radius a and total mass M. 


Everywhere except on the ring the potential u(r) satisfies the Laplace equation, and so if 
we use polar coordinates with the normal to the ring as polar axis, as in figure 21.9, a 
solution of the form (21.58) can be assumed. 

We expect the potential u(r, 0, p) to tend to zero as r > «, and also to be finite at r = 0. 
At first sight this might seem to imply that all A and B, and hence u, must be identically 
zero, an unacceptable result. In fact, what it means is that different expressions must apply 
to different regions of space. On the ring itself we no longer have V7u = 0 and so it is not 
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surprising that the form of the expression for u changes there. Let us therefore take two 
separate regions. 
In the region r >a 


(i) we must have u > 0 as r > ©, implying that all A = 0, and 
(ii) the system is axially symmetric and so only m = 0 terms appear. 


With these restrictions we can write as a trial form 
u(r, 0,6) = > Ber) P?(cos 8). (21.59) 
7=0 


The constants B; are still to be determined; this we do by calculating directly the potential 
where this can be done simply — in this case, on the polar axis. 

Considering a point P on the polar axis at a distance z (> a) from the plane of the ring 
(taken as 0 = 2/2), all parts of the ring are at a distance (z? + a’)'/? from it. The potential 
at P is thus straightforwardly 

GM 
“(22 + a)” 
where G is the gravitational constant. This must be the same as (21.59) for the particular 
values r = z, 0 = 0, and ¢ undefined. Since P?(cos 0) = P/(cos 0) with P/(1) = 1, putting 
r =z in (21.59) gives 


u(z,0,) = (21.60) 


oa) 


B, 
u(z,0,0) = 0 a5- (21.61) 


/=0 


However, expanding (21.60) for z > a (as it applies to this region of space) we obtain 


u(2.0,6) = 1 5(2)' +3 (2) ---], 


which on comparison with (21.61) gives’ 


Bo = —GM, 
GMa¥(—1)(2f — 1)! 
By = tora, (21.62) 
Boy = 0. 


We now conclude the argument by saying that if a solution for a general point (r, 0, d) 
exists at all, which of course we very much expect on physical grounds, then it must be 
(21.59) with the B, given by (21.62). This is so because thus defined it is a function with 
no arbitrary constants and which satisfies all the boundary conditions, and the uniqueness 
theorem states that there is only one such function. The expression for the potential in the 
region r > a is therefore 


00 aa if a 
u(r, 0.) = ou o> cyrer un Ge rates 
2. /| 


The expression for r < a can be found in a similar way. The finiteness of u at r = 0 and 
the axial symmetry give 


u(r, 0,6) = > Arr’ P?(cos 0). 


/=0 


8 (2/1)! =1x3x--x (2-1). 
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Comparing this expression for r = z, 0 = 0 with the z < a expansion of (21.60), which is 
valid for any z, establishes Ax/+; = 0, Ao = —GM/a and 

GM (-1)(2¢—-1)!! 

ee 


so that the final expression valid, and convergent, for r < a is thus 


GM “ (-1)/(2¢ — 1)! 20 
u(r, 0, ) = -—— 1+ eee (5) Pateos|. 
aa /| 


It is easy to check that the solution obtained has the expected physical value for large r 
and for r = 0 and is continuous at r =a. < 


21.3.4 Separation of variables for inhomogeneous equations 


So far our discussion of the method of separation of variables has been limited 
to the solution of homogeneous equations such as the Laplace equation and the 
wave equation. The solutions of inhomogeneous PDEs are usually obtained using 
the Green’s function methods to be discussed below in section 21.5. However, as a 
final illustration of the usefulness of the separation of variables, we now consider 
its application to the solution of inhomogeneous equations. 

Because of the added complexity in dealing with inhomogeneous equations, we 
shall restrict our discussion to the solution of Poisson’s equation, 


V-u = p(n), (21.63) 


in spherical polar coordinates, although the general method can accommodate 
other coordinate systems and equations. In physical problems the RHS of (21.63) 
usually contains some multiplicative constant(s). If u is the electrostatic potential 
in some region of space in which p is the density of electric charge then V7u = 
—p(r)/eo. Alternatively, u might represent the gravitational potential in some 
region where the matter density is given by p, so that V7u = 42Gp(r). 

We will simplify our discussion by assuming that the required solution wu is 
finite on the polar axis and also that the system possesses axial symmetry about 
that axis — in which case p does not depend on the azimuthal angle ¢. The key 
to the method is then to assume a separated form for both the solution u and the 
density term p. 

From the discussion of Laplace’s equation, for systems with axial symmetry 
only m = 0 terms appear, and so the angular part of the solution can be 
expressed in terms of Legendre polynomials P,(cos @). Since these functions form 
an orthogonal set let us expand both u and p in terms of them: 


u= s R,(r)P;(cos 0), (21.64) 
7=0 

p= = F,(r)P;(cos 0), (21.65) 
¢=0 
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where the coefficients R,(r) and F,(r) in the Legendre polynomial expansions 
are functions of r. Since in any particular problem p is given, we can find the 
coefficients F/(r) in the expansion in the usual way (see subsection 18.1.2). It then 
only remains to find the coefficients R/(r) in the expansion of the solution u. 

Writing V? in spherical polars and substituting (21.64) and (21.65) into (21.63) 
we obtain 


— [P-(cos 0) d paRe\ , Rd ( pdPr(cosd)\| _ * 
Ss | r2 dr dr ' 7-2 sin 0 dO ae do d» F¢(r)Pe(cos 8). 


7=0 
(21.66) 


However, if, in equation (21.44) of our discussion of the angular part of the 
solution to Laplace’s equation, we set m = 0 we conclude that 
Le! df gdPelcos 0) 
— — | sind ———— 
sin 0 d0 do 
Substituting this into (21.66), we find that the LHS is greatly simplified and we 
obtain 


) = —/¢(¢ + 1)P/(cos 6). 


> [44 u G we) ee P,(cos 4) = Far )P,(cos 0). 


7=0 ¢7=0 


This relation is most easily satisfied by equating terms on both sides for each 
value of ¢ separately, so that for 7 = 0,1,2,... we have 
1d ( ,dR, Ce + 1)R, 
ee. yp ee ES Rr). 21.67 
r? dr (: dr ) r2 cr) ( ) 
This is an ODE in which F;,(r) is given, and it can therefore be solved for 
R/(r). The solution to Poisson’s equation, u, is then obtained by making the 
superposition (21.64). 


>In a certain system, the electric charge density p is distributed as follows: 


={" for0<r<a, 


0 forr>a. 


Find the electrostatic potential inside and outside the charge distribution, given that both 
the potential and its radial derivative are continuous everywhere. 


The electrostatic potential u satisfies 


Vu = —(A/eo)\rcos@ forO0<r<a, 
4u= 0 forr >a. 


For r < ‘ the RHS can be written —(A/eo)rP\(cos 0), and the coefficients in (21.65) are 
simply F\(r) = —(Ar/eo) and F/(r) = 0 for ¢ # 1. Therefore we need only calculate R,(r), 
which ae (21.67) for 7 = 1: 

fg dR, 2R, Ar 
Pdr \" ad 


r2 €0 ‘ 
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This can be rearranged to give 
Ul - A : 
PR! +2rRi —2R;) =—-—, 
€0 
where the prime denotes differentiation with respect to r. The LHS is homogeneous and 
the equation can be reduced by the substitution r = expt, and writing R,(r) = S(t), to 


S4+S$-28= ae exp 3t, (21.68) 
€0 


where the dots indicate differentiation with respect to f. 
This is an inhomogeneous second-order ODE with constant coefficients and can be 
straightforwardly solved by the methods of subsection 15.2.1 to give 


A 
S(t) = c, expt + cp exp(—2t) — —— exp 3t. 
10€o 
Recalling that r = expt we find 


A 
Rr) = qr tor? — —r. 
u(r) = 2 10e0 
Since we are interested in the region r < a we must have c7 = 0 for the solution to remain 
finite. Thus inside the charge distribution the electrostatic potential has the form 


u(r,0, ob) = (ar - a ‘) P(cos 0). (21.69) 

Outside the charge distribution (for r > a), however, the electrostatic potential obeys 
Laplace’s equation, Vu = 0, and so given the symmetry of the problem and the requirement 
that u > oo as r — oo the solution must take the form 


uo(r, 0, ) = FA Pi(cos 0). (21.70) 
/=0 


We can now use the boundary conditions at r = a to fix the constants in (21.69) and 
(21.70). The requirement of continuity of the potential and its radial derivative at r =a 
imply that 


u1(a, 0, b) = up(a, 0, >), 
a ae 


Gp (90) = 


Clearly B, = 0 for ¢ # 1; carrying out the necessary differentiations and setting r = a in 
(21.69) and (21.70) we obtain the simultaneous equations 


A ;, B 
cya — ioe” re, 
oy 34 gp = 2B 

10€ a ° 


which may be solved to give cy = Aa?/(6e€9) and B, = Aa>/(15e9). Since P;(cos 0) = cos 0, 
the electrostatic potentials inside and outside the charge distribution are given, respectively, 
by 


ar Pr Aa? cos 0 
u(r, 0,6) = a(¢- 5) cos 0, ur(r, 0, d) = isa ge | 


746 


21.4 INTEGRAL TRANSFORM METHODS 


21.4 Integral transform methods 


In the method of separation of variables our aim was to keep the independent 
variables in a PDE as separate as possible. We now discuss the use of integral 
transforms in solving PDEs, a method by which one of the independent variables 
can be eliminated from the differential coefficients. It will be assumed that the 
reader is familiar with Laplace and Fourier transforms and their properties, as 
discussed in chapter 13. 

The method consists simply of transforming the PDE into one containing 
derivatives with respect to a smaller number of variables. Thus, if the original 
equation has just two independent variables, it may be possible to reduce the 
PDE into a soluble ODE. The solution obtained can then (where possible) be 
transformed back to give the solution of the original PDE. As we shall see, 
boundary conditions can usually be incorporated in a natural way. 

Which sort of transform to use, and the choice of the variable(s) with respect 
to which the transform is to be taken, is a matter of experience; we illustrate this 
in the example below. In practice, transforms can be taken with respect to each 
variable in turn, and the transformation that affords the greatest simplification 
can be pursued further. 


>A semi-infinite tube of constant cross-section contains initially pure water. At time t = 0, 
one end of the tube is put into contact with a salt solution and maintained at a concentration 
uo. Find the total amount of salt that has diffused into the tube after time t, if the diffusion 
constant is Kk. 


The concentration u(x,t) at time t and distance x from the end of the tube satisfies the 
diffusion equation 

Ou du 

at OP (21.71) 
which has to be solved subject to the boundary conditions u(0,t) = uo for all t and 
u(x,0) = 0 for all x > 0. 

Since we are interested only in t > 0, the use of the Laplace transform is suggested. 
Furthermore, it will be recalled from chapter 13 that one of the major virtues of Laplace 
transformations is the possibility they afford of replacing derivatives of functions by simple 
multiplication by a scalar. If the derivative with respect to time were so removed, equation 
(21.71) would contain only differentiation with respect to a single variable. Let us therefore 
take the Laplace transform of (21.71) with respect to ¢: 


i Ka exp(—st) dt | AG exp(—st) dt. 


On the LHS the (double) differentiation is with respect to x, whereas the integration is 
with respect to the independent variable t. Therefore the derivative can be taken outside 
the integral. Denoting the Laplace transform of u(x,t) by a(x,s) and using result (13.57) 
to rewrite the transform of the derivative on the RHS (or by integrating directly by parts), 
we obtain 


Cn _ 
Kaa = su(x,s) — u(x, 0). 


But from the boundary condition u(x,0) = 0 the last term on the RHS vanishes, and the 
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solution is immediate: 


u(x,s) = Aexp (2 x) + Bexp (-/= x) F 


where the constants A and B may depend on s. 

We require u(x,t) > 0 as x — o and so we must also have u(co,s) = 0; consequently 
we require that A = 0. The value of B is determined by the need for u(0,t) = uo and hence 
that 


u(0,s) = i; . Ug exp(—st) dt = =. 
0 ! 


We thus conclude that the appropriate expression for the Laplace transform of u(x, t) is 


u s 
ii(x,s) = — exp (- ee x) (21.72) 
s K 

To obtain u(x,t) from this result requires the inversion of this transform — a task that is 
generally difficult and requires a contour integration. This is discussed in chapter 24, but 
for completeness we note that the solution is 


u(x, t) = uo i! ~e (=) ; 


where erf(x) is the error function discussed in the Appendix. (The more complete sets of 
mathematical tables list this inverse Laplace transform.) 

In the present problem, however, an alternative method is available. Let w(t) be the 
amount of salt that has diffused into the tube in time f; then 


w(t) = Ve u(x, t) dx, 


and its transform is given by 


W(s) = fe dt exp(—st) Ve u(x, t) dx 


-| ax u(x, t) exp(—st) dt 
0 0 


= | u(x, s) dx. 


Substituting for a(x,s) from (21.72) into the last integral and integrating, we obtain 


W(s) = uok!/?573/?, 


This expression is much simpler to invert, and referring to the table of standard Laplace 
transforms (table 13.1) we find 


w(t) = 2(x/m)"?ugt!?, 


which is thus the required expression for the amount of diffused salt at time t. < 


The above example shows that in some circumstances the use of a Laplace 
transformation can greatly simplify the solution of a PDE. However, it will have 
been observed that (as with ODEs) the easy elimination of some derivatives is 
usually paid for by the introduction of a difficult inverse transformation. This 
problem, although still present, is less severe for Fourier transformations. 
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> An infinite metal bar has an initial temperature distribution f(x) along its length. Find 
the temperature distribution at a later time t. 


We are interested in values of x from —co to 00, which suggests Fourier transformation 
with respect to x. Assuming that the solution obeys the boundary conditions u(x,t) > 0 
and du/dx — 0 as |x| > o, we may Fourier-transform the one-dimensional diffusion 
equation (21.71) to obtain 


eae (x, t) 1 


xp(—ikx) dx = g / u(x, t) exp(—ikx) dx, 


i Tai 


where on the RHS we have taken the partial derivative with respect to t outside the 
integral. Denoting the Fourier transform of u(x,t) by u(k,t), and using equation (13.28) to 
rewrite the Fourier transform of the second derivative on the LHS, we then have 


ou(k, t) 
ot 
This first-order equation has the simple solution 
ak, t) = u(k, 0) exp(—Kk7t), 


where the initial conditions give 


—Kkik,t) = 


u(k, 0) = Te a u(x, 0) exp(—ikx) dx 
ane i f(x) exp(—ikx) dx = f(k). 


st Ja 


Thus we may write the Fourier transform of the solution as 
i(k, t) = f (k) exp(—Kk7t) = J2r f (k)G(k, 0), (21.73) 


where we have defined the function G(k, t) = (./2n)~! exp(—xk?t). Since ti(k,t) can be 
written as the product of two Fourier transforms, we can use the convolution theorem, 
subsection 13.1.7, to write the solution as 


u(x,t) = if G(x — x’, t)f (x’) dx 


where G(x, f) is the Green’s function for this problem (see subsection 15.2.5). This function 
is the inverse Fourier transform of G(k,t) and is thus given by 


G(x, t) = x i exp(—Kk’t) exp(ikx) dk 


1. f% pe DS 
oe fe exp | kt (i *x)| dk. 


Completing the square in the integrand we find 


1 x? x ix \? 
G(x, t) = a exp (- iz) hs exp x (: - =) | dk 
1 x i. 2 q 
Bae exp (-=) se exp (—ntk ) dk 
x 


where in the second line we have made the substitution k’ = k — ix/(2xt), and in the last 
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y} 
_Z/) 


Figure 21.10 Diffusion of heat from a point source in a metal bar: the curves 
show the temperature u at position x for various times t; < t) < t3. The area 
under the curves remains constant, since the total heat energy is conserved. 


| 
\ 


x=da x 


line we have used the standard result for the integral of a Gaussian, given in subsection 
6.4.2. (Strictly speaking the change of variable from k to k’ shifts the path of integration 
off the real axis, since k’ is complex for real k, and so results in a complex integral, as will 
be discussed in chapter 24. Nevertheless, in this case the path of integration can be shifted 
back to the real axis without affecting the value of the integral.) 

Thus the temperature in the bar at a later time ¢ is given by 


nae! - Se aay 
u(x,t) = al. exp =| f(x’) dx’, (21.74) 


which may be evaluated (numerically if necessary) when the form of f(x) is given. < 


As we might expect from our discussion of Green’s functions in chapter 15, 
we see from (21.74) that, if the initial temperature distribution is f(x) = 6(x — a), 
Le. a ‘point’ source at x = a, then the temperature distribution at later times is 


simply given by 
ey 


u(x, t) = G(x —a,t) = ree 


1 
V4ankt 
The temperature at several later times is illustrated in figure 21.10, which shows 
that the heat diffuses out from its initial position; the width of the Gaussian 
increases as ,/t, a dependence on time which is characteristic of diffusion processes. 

The reader may have noticed that in both examples using integral transforms 
the solutions have been obtained in closed form — albeit in one case in the form 
of an integral. This differs from the infinite series solutions usually obtained via 
the separation of variables. It should be noted that this behaviour is a result of 
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the infinite range in x, rather than of the transform method itself. In fact the 
method of separation of variables would yield the same solutions, since in the 
infinite-range case the separation constant is not restricted to take on an infinite 
set of discrete values but may have any real value, with the result that the sum 
over A becomes an integral, as mentioned at the end of section 21.2. 


> An infinite metal bar has an initial temperature distribution f(x) along its length. Find 
the temperature distribution at a later time t using the method of separation of variables. 


This is the same problem as in the previous example, but we now seek a solution by 
separating variables. From (21.12) a separated solution for the one-dimensional diffusion 
equation is given by 

u(x,t) = [A exp(idx) + B exp(—idx)] exp(—K72), 
where —/? is the separation constant. Since the bar is infinite we do not require the 
solution to take a given form at any finite value of x (for instance at x = 0) and so there 
is no restriction on 4 other than its being real. Therefore instead of the superposition of 


such solutions in the form of a sum over allowed values of 2 we have an integral over 
all 2, 


u(x, t) = A(A) exp(—K/7t) exp(idx) dA, (21.75) 


vel 
where in taking 2 from —oo to co we need include only one of the complex exponentials; 
we have taken a factor 1//2n out of A(A) for convenience. We can see from (21.75) 
that the expression for u(x,t) has the form of an inverse Fourier transform (where 2 is 
the transform variable). Therefore, Fourier-transforming both sides and using the Fourier 
inversion theorem, we find 

U(A, t) = A(A) exp(—K7t). 


Now, the initial boundary condition requires 


oe) 


ala 


from which, using the Fourier inversion theorem once more, we see that A(A) = f(A). 
Therefore we have 


u(x, 0) = A) exp(iAx) dd = f(x), 


(A, t) = f(A) exp(—K228), 


which is identical to (21.73) in the previous example (but with k replaced by 2), and hence 
leads to the same result. < 


21.5 Inhomogeneous problems — Green’s functions 


In chapters 15 and 17 we encountered Green’s functions and found them a useful 
tool for solving inhomogeneous linear ODEs. We now discuss their usefulness in 
solving inhomogeneous linear PDEs. 

For the sake of brevity we shall again denote a linear PDE by 


Lu(r) = p(n), (21.76) 


where C£ is a linear partial differential operator. For example, in Laplace’s equation 
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we have £L = V’, whereas for Helmholtz’s equation £ = V?+k*. Note that we have 
not specified the dimensionality of the problem, and (21.76) may, for example, 
represent Poisson’s equation in two or three (or more) dimensions. The reader 
will also notice that for the sake of simplicity we have not included any time 
dependence in (21.76). Nevertheless, the following discussion can be generalised 
to include it. 

As we discussed in subsection 20.3.2, a problem is inhomogeneous if the fact 
that u(r) is a solution does not imply that any constant multiple Au(r) is also a 
solution. This inhomogeneity may derive from either the PDE itself or from the 
boundary conditions imposed on the solution. 

In our discussion of Green’s function solutions of inhomogeneous ODEs (see 
subsection 15.2.5) we dealt with inhomogeneous boundary conditions by making a 
suitable change of variable such that in the new variable the boundary conditions 
were homogeneous. In an analogous way, as illustrated in the final example 
of section 21.2, it is usually possible to make a change of variables in PDEs to 
transform between inhomogeneity of the boundary conditions and inhomogeneity 
of the equation. Therefore let us assume for the moment that the boundary 
conditions imposed on the solution u(r) of (21.76) are homogeneous. This most 
commonly means that if we seek a solution to (21.76) in some region V then 
on the surface S that bounds V the solution obeys the conditions u(r) = 0 or 
ou/0n = 0, where du/Oén is the normal derivative of u at the surface S. 

We shall discuss the extension of the Green’s function method to the direct so- 
lution of problems with inhomogeneous boundary conditions in subsection 21.5.2, 
but we first highlight how the Green’s function approach to solving ODEs can 
be simply extended to PDEs for homogeneous boundary conditions. 


21.5.1 Similarities to Green’s functions for ODEs 


As in the discussion of ODEs in chapter 15, we may consider the Green’s 
function for a system described by a PDE as the response of the system to a ‘unit 
impulse’ or ‘point source’. Thus if we seek a solution to (21.76) that satisfies some 
homogeneous boundary conditions on u(r) then the Green’s function G(r,ro) for 
the problem is a solution of 


LG(r,ro) = O(r — ro), (21.77) 


where ro lies in V. The Green’s function G(r,ro) must also satisfy the imposed 
(homogeneous) boundary conditions. 

It is understood that in (21.77) the £ operator expresses differentiation with 
respect to r as opposed to rp. Also, 6(r — ro) is the Dirac delta function (see 
chapter 13) of dimension appropriate to the problem; it may be thought of as 
representing a unit-strength point source at r = fro. 

Following an analogous argument to that given in subsection 15.2.5 for ODEs, 
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if the boundary conditions on u(r) are homogeneous then a solution to (21.76) 
that satisfies the imposed boundary conditions is given by 


u(t) = iy G(x, t0)0(t0) dV (to), (21.78) 


where the integral on ro is over some appropriate ‘volume’. In two or more 
dimensions, however, the task of finding directly a solution to (21.77) that satisfies 
the imposed boundary conditions on S can be a difficult one, and we return to 
this in the next subsection. 

An alternative approach is to follow a similar argument to that presented in 
chapter 17 for ODEs and so to construct the Green’s function for (21.76) as a 
superposition of eigenfunctions of the operator £, provided £ is Hermitian. By 
analogy with an ordinary differential operator, a partial differential operator is 
Hermitian if it satisfies 


* 


[ewemmar = / w (nicer dy ‘ 
Vv Vv 


where the asterisk denotes complex conjugation and v and w are arbitrary func- 
tions obeying the imposed (homogeneous) boundary condition on the solution of 
Lu(r) = 0. 

The eigenfunctions u,(r), 1 = 0,1,2,..., of £ satisfy 


Lu,(r) = AnUn(t), 


where 4, are the corresponding eigenvalues, which are all real for an Hermitian 
operator £. Furthermore, each eigenfunction must obey any imposed (homo- 
geneous) boundary conditions. Using an argument analogous to that given in 
chapter 17, the Green’s function for the problem is given by 


a 


G(r.) = >> Mn(r)un(to) (21.79) 
An 
n=0 
From (21.79) we see immediately that the Green’s function (irrespective of how 
it is found) enjoys the property 
G(r, to) = G" (ro, 4). 


Thus, if the Green’s function is real then it is symmetric in its two arguments. 

Once the Green’s function has been obtained, the solution to (21.76) is again 
given by (21.78). For PDEs this approach can become very cumbersome, however, 
and so we shall not pursue it further here. 


21.5.2 General boundary-value problems 


As mentioned above, often inhomogeneous boundary conditions can be dealt 
with by making an appropriate change of variables, such that the boundary 
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conditions in the new variables are homogeneous although the equation itself is 
generally inhomogeneous. In this section, however, we extend the use of Green’s 
functions to problems with inhomogeneous boundary conditions (and equations). 
This provides a more consistent and intuitive approach to the solution of such 
boundary-value problems. 

For definiteness we shall consider Poisson’s equation 


V-u(r) = p(r), (21.80) 


but the material of this section may be extended to other linear PDEs of the form 
(21.76). Clearly, Poisson’s equation reduces to Laplace’s equation for p(r) = 0 and 
so our discussion is equally applicable to this case. 

We wish to solve (21.80) in some region V bounded by a surface S, which may 
consist of several disconnected parts. As stated above, we shall allow the possibility 
that the boundary conditions on the solution u(r) may be inhomogeneous on S, 
although as we shall see this method reduces to those discussed above in the 
special case that the boundary conditions are in fact homogeneous. 

The two common types of inhomogeneous boundary condition for Poisson’s 
equation are (as discussed in subsection 20.6.2): 


(i) Dirichlet conditions, in which u(r) is specified on S, and 
(ii) Neumann conditions, in which du/0n is specified on S. 


In general, specifying both Dirichlet and Neumann conditions on S overdetermines 
the problem and leads to there being no solution. 

The specification of the surface S requires some further comment, since S$ 
may have several disconnected parts. If we wish to solve Poisson’s equation 
inside some closed surface S then the situation is straightforward and is shown 
in figure 21.11(a). If, however, we wish to solve Poisson’s equation in the gap 
between two closed surfaces (for example in the gap between two concentric 
conducting cylinders) then the volume V is bounded by a surface S that has 
two disconnected parts S; and S2, as shown in figure 21.11(b); the direction of 
the normal to the surface is always taken as pointing out of the volume V. A 
similar situation arises when we wish to solve Poisson’s equation outside some 
closed surface S;. In this case the volume V is infinite but is treated formally 
by taking the surface S) as a large sphere of radius R and letting R tend to 
infinity. 

In order to solve (21.80) subject to either Dirichlet or Neumann boundary 
conditions on S, we will remind ourselves of Green’s second theorem, equation 
(11.20), which states that, for two scalar functions ¢(r) and y(r) defined in some 
volume V bounded by a surface S, 


[ore —ypV°o)dV = [ow —pV¢)-fidS, (21.81) 
V S 
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— 


= 


(a) (b) 


Figure 21.11 Surfaces used for solving Poisson’s equation in different 
regions V. 


where on the RHS it is common to write, for example, Vy - dS as (dy/dn) dS. 
The expression dy/0n stands for Vp - fi, the rate of change of p in the direction 
of the unit outward normal fi to the surface S. 

The Green’s function for Poisson’s equation (21.80) must satisfy 


V’G(r, ro) = d(r — ro), (21.82) 


where ro lies in V. (As mentioned above, we may think of G(r, ro) as the solution 
to Poisson’s equation for a unit-strength point source located at r = ro.) Let us 
for the moment impose no boundary conditions on G(r, ro). 

If we now let @ = u(r) and yw = G(r,ro) in Green’s theorem (21.81) then we 
obtain 


/ [u(r)VG(r,r0) — G(r,r0) V2u(r)] dV (r) 
V 


-| ue) EO — Gry) SO] dS(r), 
s n 


on 


where we have made explicit that the volume and surface integrals are with 
respect to r. Using (21.80) and (21.82) the LHS can be simplified to give 


if [u(e)5(r — to) — G(r, 0)p(n)] AV) 


_ / ue) — Gtr) AO dS(r). (21.83) 
s on on 


) 


Since ro lies within the volume V, 
[wns = 10) V(r) = ut), 
Vv 
and thus on rearranging (21.83) the solution to Poisson’s equation (21.80) can be 
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written as 


OG(r, ro) Ou(r) 


u(ro) = [ campnavin+ [ lu) aa — G(r, 10) = nm 


dS(r). 
(21.84) 


Clearly, we can interchange the roles of r and ro in (21.84) if we wish. (Remember 
also that, for a real Green’s function, G(r, 19) = G(ro, r).) 

Equation (21.84) is central to the extension of the Green’s function method 
to problems with inhomogeneous boundary conditions, and we next discuss its 
application to both Dirichlet and Neumann boundary-value problems. But, before 
doing so, we also note that if the boundary condition on S is in fact homogeneous, 
so that u(r) = 0 or du(r)/dn = 0 on S, then demanding that the Green’s function 
G(r,ro) also obeys the same boundary condition causes the surface integral in 
(21.84) to vanish, and we are left with the familiar form of solution given in 
(21.78). The extension of (21.84) to a PDE other than Poisson’s equation is 
discussed in exercise 21.28. 


21.5.3 Dirichlet problems 


In a Dirichlet problem we require the solution u(r) of Poisson’s equation (21.80) 
to take specific values on some surface S that bounds V, ie. we require that 
u(r) = f(r) on S where f is a given function. 

If we seek a Green’s function G(r,ro) for this problem it must clearly satisfy 
(21.82), but we are free to choose the boundary conditions satisfied by G(r,ro) in 
such a way as to make the solution (21.84) as simple as possible. From (21.84), 
we see that by choosing 


G(t,ro) =0 forronS (21.85) 


the second term in the surface integral vanishes. Since u(r) = f(r) on S, (21.84) 
then becomes 


gael (r, oOo) 


u(ro) = if G(r, ro) p(r) avert [10S dS(r). (21.86) 


Thus we wish to find the Dirichlet Green’s function that 


(i) satisfies (21.82) and hence is singular at r = ro, and 
(ii) obeys the boundary condition G(r,ro) = 0 for r on S. 


In general, it is difficult to obtain this function directly, and so it is useful to 
separate these two requirements. We therefore look for a solution of the form 


G(r, ro) = F(t, ro) + A(r, ro), 


where F(r, rg) satisfies (21.82) and has the required singular character at r = ro but 
does not necessarily obey the boundary condition on S, whilst H(r,ro) satisfies 
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the corresponding homogeneous equation (i.e. Laplace’s equation) inside V but 
is adjusted in such a way that the sum G(r,ro) equals zero on S. The Green’s 
function G(r,ro) is still a solution of (21.82) since 


V°G(r, ro) = V7F(r, ro) + V7A(r, 10) = VF (1,10) + 0 = d(r — ro). 


The function F(r,ro) is called the fundamental solution and will clearly take 
different forms depending on the dimensionality of the problem. Let us first 
consider the fundamental solution to (21.82) in three dimensions. 


> Find the fundamental solution to Poisson’s equation in three dimensions that tends to zero 


as |r| > 00. 


We wish to solve 
V’F(r,¥0) = d(r — Yo) (21.87) 


in three dimensions, subject to the boundary condition F(r,ro) — 0 as |r| — oo. Since the 
problem is spherically symmetric about ro, let us consider a large sphere S of radius R 
centred on ro, and integrate (21.87) over the enclosed volume V. We then obtain 


i V-F(t,¥0) dV = | O(r —ro) dV = 1, (21.88) 
Vv Vv 
since V encloses the point ro. However, using the divergence theorem, 
[ vremar = [ VF tet): ads, (21.89) 
Vv s 


where fi is the unit normal to the large sphere S at any point. 
Since the problem is spherically symmetric about rp, we expect that 
F(t,ro) = F(|r — rol) = F(r), 


ie. that F has the same value everywhere on S. Thus, evaluating the surface integral in 
(21.89) and equating it to unity from (21.88), we haves 


Integrating this expression we obtain 
1 
F(r) = ——— + constant, 
4ur 
but, since we require F(r,1o) — 0 as |r| > 00, the constant must be zero. The fundamental 
solution in three dimensions is consequently given by 


1 


F(r,r0) = -———_-. 
(r,10) 4n|r — ro| 


(21.90) 


This is clearly also the full Green’s function for Poisson’s equation subject to the boundary 
condition u(r) — 0 as |r| > 00. < 


Using (21.90) we can write down the solution of Poisson’s equation to find, 
S A vertical bar to the right of an expression is a common alternative to enclosing the expression in 


square brackets; as usual, the subscript shows the value of the variable at which the expression is 
to be evaluated. 
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for example, the electrostatic potential u(r) due to some distribution of electric 
charge p(r). The electrostatic potential satisfies 


V-u(r) = paay 
€0 
where u(r) —> 0 as |r| > oo. Since the boundary condition on the surface at 
infinity is homogeneous the surface integral in (21.86) vanishes, and using (21.90) 
we recover the familiar solution 


p(t) 
= | ————-dV 21.91 
ure) = [| <A avie, (21.91) 
where the volume integral is over all space. 

We can develop an analogous theory in two dimensions. As before the funda- 
mental solution satisfies 


V-F(r, 10) = 5(r — ro), (21.92) 


where 6(r—ro) is now the two-dimensional delta function. Following an analogous 
method to that used in the previous example, we find the fundamental solution 
in two dimensions to be given by 


F(r,ro) = = In |r — ro| + constant. (21.93) 


From the form of the solution we see that in two dimensions we cannot apply 
the condition F(r,ro) — 0 as |r| > oo, and in this case the constant does not 
necessarily vanish. 

We now return to the task of constructing the full Dirichlet Green’s function. To 
do so we wish to add to the fundamental solution a solution of the homogeneous 
equation (in this case Laplace’s equation) such that G(r,ro) = 0 on S, as required 
by (21.86) and its attendant conditions. The appropriate Green’s function is 
constructed by adding to the fundamental solution ‘copies’ of itself that represent 
‘image’ sources at different locations outside V. Hence this approach is called the 
method of images. 

In summary, if we wish to solve Poisson’s equation in some region V subject to 
Dirichlet boundary conditions on its surface S then the procedure and argument 
are as follows. 


(i) To the single source 6(r — ro) inside V add image sources outside V 
N 
by qno(t —¥,) with r, outside V, 
n=1 


where the positions r, and the strengths gq, of the image sources are to be 
determined as described in step (iti) below. 
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(ii) Since all the image sources lie outside V, the fundamental solution cor- 
responding to each source satisfies Laplace’s equation inside V. Thus we 
may add the fundamental solutions F(r,r,) corresponding to each image 
source to that corresponding to the single source inside V, obtaining the 
Green’s function 

N 
G(r,%o) = F(t,ro) + ye GnF (t,¥n). 
n=1 

(iti) Now adjust the positions r, and strengths q, of the image sources so 
that the required boundary conditions are satisfied on S. For a Dirichlet 
Green’s function we require G(r,ro) = 0 for r on S. 

(iv) The solution to Poisson’s equation subject to the Dirichlet boundary 
condition u(r) = f(r) on S is then given by (21.86). 


In general it is very difficult to find the correct positions and strengths for the 
images, ic. to make them such that the boundary conditions on S are satisfied. 
Nevertheless, it is possible to do so for certain problems that have simple geometry. 
In particular, for problems in which the boundary S consists of straight lines (in 
two dimensions) or planes (in three dimensions), positions of the image points 
can be deduced simply by imagining the boundary lines or planes to be mirrors 
in which the single source in V (at ro) is reflected. 


> Solve Laplace’s equation Vu = 0 in three dimensions in the half-space z > 0, given that 
u(r) = f(r) on the plane z = 0. 


The surface S bounding V consists of the xy-plane and the surface at infinity. Therefore, 
the Dirichlet Green’s function for this problem must satisfy G(r,ro) = 0 on z = 0 and 
G(r, ro) > O as |r| — oo. Thus it is clear in this case that we require one image source at a 
position r, that is the reflection of ro in the plane z = 0, as shown in figure 21.12 (so that 
r, lies in z < 0, outside the region in which we wish to obtain a solution). It is also clear 
that the strength of this image should be —1. 

Therefore by adding the fundamental solutions corresponding to the original source 
and its image we obtain the Green’s function 


uf 1 
4n|r—ro| 4z|r —4r,|’ 


G(r, ro) (21.94) 
where r; is the reflection of ro in the plane z = 0, Le. if ro = (Xo, yo, Zo) then ry = (Xo, Yo, —Zo)- 
Clearly G(r,1ro) — 0 as |r| > o as required. Also G(r,ro) = 0 on z = 0, and so (21.94) is 
the desired Dirichlet Green’s function. 

The solution to Laplace’s equation is then given by (21.86) with p(r) = 0, 


0 
ure) = [ fy 


Clearly the surface at infinity makes no contribution to this integral. The outward-pointing 
unit vector normal to the xy-plane is simply fi = —k (where k is the unit vector in the 
z-direction), and so 


dS(r). (21.95) 


OG(r, ro) OG(r, ro) = AE PGs 


on Oz 
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Figure 21.12 The arrangement of images for solving Laplace’s equation in 


the half-space z > 0. 


We may evaluate this normal derivative by writing the Green’s function (21.94) explicitly 
in terms of x, y and z (and xo, yo and zo) and calculating the partial derivative with respect 


to z directly. It is usually quicker, however, to 
V\r —ro| = 


thus 
r—fro 


VG(r, ro) = 
Since rp = (Xo, Yo, Zo) and r = (Xo, Yo, —Zo) the 
OG(r, ro) 

Oz 


Z— 


use the fact thatS 
r—Yro . 


lr—ro|’ 


| Nae 


~ 4nir—noP 4ayr—ni) 


normal derivative is given by 


= —k- VG(r, ro) 


Zo Z+2Zo 


~ 4gfr—noe | 4air—n [2 


(21.96) 


Therefore on the surface z = 0, and writing out the dependence on x, y and z explicitly, 


we have 
OG(r, ro) 


22 


Oz 


zo 40 [(x — x0)? +(y — yo)? + 6)?" 


Inserting this expression into (21.95) we obtain the solution 


2g. of SE f(x,y) 
S28 ix dy. 
u(Xo, Yo; 20) on [. pe [(& — x0)? + (y — yo)? + 2B)? x dy. <4 


An analogous procedure may be applied in two-dimensional problems. For 


S Since |r — ro|? = (r — ro) - (r — ro) we have V|r — ro|* = 2(r — ro), from which we obtain 


V(r 


1 2%r— = 
eee (r — ro) r—ro 


2 (|r 


—ro)?)/2 r= rol’ 


Note that this result holds in two and three dimensions. 
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example, in solving Poisson’s equation in two dimensions in the half-space x > 0 
we again require just one image charge, of strength qi = —1, at a position r; that 
is the reflection of ro in the line x = 0. Since we require G(r,ro) = 0 when r lies 
on x = 0, the constant in (21.93) must equal zero, and so the Dirichlet Green’s 
function is 


1 
G(r, to) = 5— (In|r ¥o| — In |r —rj|) . 


Clearly G(r,ro) tends to zero as |r| > 00. If, however, we wish to solve the two- 
dimensional Poisson equation in the quarter space x > 0, y > 0, then more image 
points are required. 


>A line charge in the z-direction of charge density 2 is placed at some position Yo in the 
quarter-space x > 0, y > 0. Calculate the force per unit length on the line charge due to 
the presence of thin earthed plates along x =0 and y = 0. 


Here we wish to solve Poisson’s equation, 


Vu = —5(r — 10), 
€0 
in the quarter space x > 0, y > 0. It is clear that we require three image line charges 
with positions and strengths as shown in figure 21.13 (all of which lie outside the region 
in which we seek a solution). The boundary condition that the electrostatic potential u is 
zero on x = 0 and y = 0 (shown as the ‘curve’ C in figure 21.13) is then automatically 
satisfied, and so this system of image charges is directly equivalent to the original situation 
of a single line charge in the presence of the earthed plates along x = 0 and y = 0. Thus 
the electrostatic potential is simply equal to the Dirichlet Green’s function 


u(r) = G(r, ro) 


(In |r — ro| — In |r — ry] + In |r — r2| — In |r —45]) , 
2n€o 
which equals zero on C and on the ‘surface’ at infinity. 
The force on the line charge at ro, therefore, is simply that due to the three line charges 
at r, r2. and r3. The elecrostatic potential due to a line charge at r;, i = 1, 2 or 3, is given 
by the fundamental solution 


u(r) = pe In|r—ri| +c, 
2n€o 
the upper or lower sign being taken according to whether the line charge is positive or 


negative, respectively. Therefore the force per unit length on the line charge at ro, due to 
the one at rj, is given by 


r=IQ 


Adding the contributions from the three image charges shown in figure 21.13, the total 
force experienced by the line charge at ro is given by 


Va To -T) To — 12 To —T3 
2 + 2 va es 
2neo \ |to— il? [to —F2|? [Fo — 83]? 


where, from the figure, rp — 1, = 2yoj, fo — 2 = 2Xoi + 2yoj and ro — 13 = 2Xoi. Thus, in 


761 


PDES: SEPARATION OF VARIABLES AND OTHER METHODS 


Vy 
—A +A 
So Sy 
y 
ae V 
= 
C x 
+A —h 
*1 ery 


Figure 21.13 The arrangement of images for finding the force on a line 
charge situated in the (two-dimensional) quarter-space x > 0, y > 0, when the 
planes x = 0 and y = 0 are earthed. 


terms of xo and yo, the total force on the line charge due to the charge induced on the 
plates is given by 


2 1. 2xit2yj 1. 
2n€o ( 2yo J 4x5 +4y5 2x0 ) 
eet (A Nes 
Aneo(x5 + ¥5) \X0 Yo 

Further generalisations are possible. For instance, solving Poisson’s equation in 
the two-dimensional strip —oo < x < «~, 0 < y < b requires an infinite series of 
image points. 

So far we have considered problems in which the boundary S consists of 
straight lines (in two dimensions) or planes (in three dimensions), in which simple 
reflections of the source at ro in these boundaries fix the positions of the image 
points. For more complicated (curved) boundaries this is no longer possible, and 
finding the appropriate position(s) and strength(s) of the image source(s) requires 
further work. 


>Use the method of images to find the Dirichlet Green’s function for solving Poisson's 
equation outside a sphere of radius a centred at the origin. 


We need to find a solution of Poisson’s equation valid outside the sphere of radius a. 
Since an image point r; cannot lie in this region, it must be located within the sphere. The 
Green’s function for this problem is therefore 

! 4 
4n\r—ro| 4a |r —44|’ 


G(r, ro) 


where |ro| > a, |r;| < a and q is the strength of the image which we have yet to determine. 
Clearly, G(r,ro) — 0 on the surface at infinity. 
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Zz +1 10 


Figure 21.14 The arrangement of images for solving Poisson’s equation 
outside a sphere of radius a centred at the origin. For a charge +1 at ro, the 
image point r, is given by (a/|ro|)’r9 and the strength of the image charge is 
—a/|ro). 


By symmetry we expect the image point r; to lie on the same radial line as the original 
source, ro, as shown in figure 21.14, and so r; = kro where k < 1. However, for a Dirichlet 
Green’s function we require G(r — ro) = 0 on |r| = a, and the form of the Green’s function 
suggests that we need 


jr—ro|oc|r—r,| for all |r| =a. (21.97) 


Referring to figure 21.14, if this relationship is to hold over the whole surface of the 
sphere, then it must certainly hold for the points A and B. We thus require 


[rol —a — [rol +a 
a—|rn| a+ ri|’ 


which reduces to |r,| = a?/|ro|. Therefore the image point must be located at the position 


a 


r= ine 
It may now be checked that, for this location of the image point, (21.97) is satisfied over 
the whole sphere. Using the geometrical result 


2 4 
agp acl gee eeee, sas 
i ee iM Ceca a 
2 
(\rol? —2r-ro+ a’) for |r| =, (21.98) 


|ro|? 


we see that, on the surface of the sphere, 


jr—r,| = 4 —ro| for |r| =a. (21.99) 
|rol 
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Therefore, in order that G = 0 at |r| = a, the strength of the image charge must be 
—a/|¥o|. Consequently, the Dirichlet Green’s function for the exterior of the sphere is 
1 a/|ro| 
G(r, t9) = —————. + ———————__.. 
(0) = Feel * Sele (@?/ ror 


For a less formal treatment of the same problem see exercise 21.22. < 


If we seek solutions to Poisson’s equation in the interior of a sphere then the 
above analysis still holds, but r and ro are now inside the sphere and the image 
r, lies outside it. 

For two-dimensional Dirichlet problems outside the circle |r| = a, we are led 
by arguments similar to those employed previously to use the same image point 
as in the three-dimensional case, namely 


ae 


= a1. (21.100) 
Iro| 


As illustrated below, however, it is usually necessary to take the image strength 
as —1 in two-dimensional problems. 


Solve Laplace’s equation in the two-dimensional region |r| < a, subject to the boundary 
condition u = f(@) on |r| =a. 


In this case we wish to find the Dirichlet Green’s function in the interior of a disc of 
radius a, so the image charge must lie outside the disc. Taking the strength of the image 
to be —1, we have 


1 1 
G(r, ro) aa Yo| ce mite, 


where r, = (a?/|ro|7)ro lies outside the disc, and c is a constant that includes the strength 
of the image charge and does not necessarily equal zero. 

Since we require G(r,ro) = 0 when |r| = a, the value of the constant c is determined, 
and the Dirichlet Green’s function for this problem is given by 
2 


a 
r Tro? ete 


G(r, to) = = (ine Yo| — In 


nfl) (21.101) 
a 


Using plane polar coordinates, the solution to the boundary-value problem can be written 
as a line integral around the circle p = a: 


stan =| 0 eels To) 


on 


0G 

fe ct ee (21.102) 
0 op p=a 
The normal derivative of the Green’s function (21.101) is given by 
OG(r, ro) 
= — -VG(r,r 
dp =i poreeeD 
r r—Yro r—-Tr, 

. : 21.103 
2n\r| (Fae FoF) 
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Using the fact that r: = (a7/|ro|?)ro and the geometrical result (21.99), we find that 
OG(r, ro) 
op 
In plane polar coordinates, r = pcosfi+ psindj and ro = pocos doi + posin doj, 
and so 


a? — |¥o|" 


~~ Qnalr — rol?" 


p=a 


OG(r, ro) tf a’ — pe 
Op | pxa 2na) a* + pi — 2apo cos(h — do) 
On substituting into (21.102), we obtain 


= a (@ — po)f(b) dd 
u(po, $o) = mal BEG? Dangcodh = da)’ (21.104) 


which is the solution to the problem. < 


21.5.4 Neumann problems 


In a Neumann problem we require the normal derivative of the solution of 
Poisson’s equation to take on specific values on some surface S that bounds V, 
Le. we require du(r)/0n = f(r) on S, where f is a given function. As we shall see, 
much of our discussion of Dirichlet problems can be immediately taken over into 
the solution of Neumann problems. 

As we proved in section 20.7 of the previous chapter, specifying Neumann 
boundary conditions determines the relevant solution of Poisson’s equation to 
within an (unimportant) additive constant. Unlike Dirichlet conditions, Neumann 
conditions impose a self-consistency requirement. In order for a solution u to exist, 
it is necessary that the following consistency condition holds: 


[tas= [vw aas = [ vuav = [ pay, (21.105) 
Ss S Vv Vv 


where we have used the divergence theorem to convert the surface integral into 
a volume integral. As a physical example, the integral of the normal component 
of an electric field over a surface bounding a given volume cannot be chosen 
arbitrarily when the charge inside the volume has already been specified (Gauss’s 
theorem). 

Let us again consider (21.84), which is central to our discussion of Green’s 
functions in inhomogeneous problems. It reads 


uit) =f Gtrroiotn avi + [ 0) — Ge) SO dS(r). 


n 


As always, the Green’s function must obey 
V?G(r, to) = d(r ~ ro), 


where fo lies in V. In the solution of Dirichlet problems in the previous subsection, 
we chose the Green’s function to obey the boundary condition G(r,ro) = 0 on S 
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and, in a similar way, we might wish to choose 0G(r,ro)/0n = 0 in the solution of 
Neumann problems. However, in general this is not permitted since the Green’s 
function must obey the consistency condition 


: 
[eo = [ Voter) Ads = [ VGte.royav =a 
son Ss V 


The simplest permitted boundary condition is therefore 


OG(r, 1 
aw ero) a for r on S, 


where A is the area of the surface S$; this defines a Neumann Green’s function. 
If we require 0u(r)/dn = f(r) on S, the solution to Poisson’s equation is given by 


u(t) = [ G(r, ro)pte) dV (0) + i: u(r) dS(r) — | G(r. ro) f(r) dS(r) 


z i: G(r, r0)p(r) dV (r) + (ule) 5 — | G(r,ro)f(r)dS(), (21.106) 


where (u(r))s is the average of u over the surface S and is a freely specifiable con- 
stant. For Neumann problems in which the volume V is bounded by a surface S at 
infinity, we do not need the (u(r))s term. For example, if we wish to solve a Neu- 
mann problem outside the unit sphere centred at the origin then r > ais the region 
V throughout which we require the solution; this region may be considered as be- 
ing bounded by two disconnected surfaces, the surface of the sphere and a surface 
at infinity. By requiring that u(r) > 0 as |r| — oo, the term (u(r))s becomes zero. 

As mentioned above, much of our discussion of Dirichlet problems can be 
taken over into the solution of Neumann problems. In particular, we may use the 
method of images to find the appropriate Neumann Green’s function. 


> Solve Laplace’s equation in the two-dimensional region |r| < a subject to the boundary 


condition du/0n = f(@) on |r| = a, with Aes f(¢) db = 0 as required by the consistency 
condition (21.105). 


Let us assume, as in Dirichlet problems with this geometry, that a single image charge is 
placed outside the circle at 
a“ 
Tr = —>5Po, 
|ro|? 


where ro is the position of the source inside the circle (see equation (21.100)). Then, from 
(21.99), we have the useful geometrical result 


jr—r,]= —olr—rol for |r| =a. (21.107) 
|ro| 


Leaving the strength q of the image as a parameter, the Green’s function has the form 


1 
G(r, 1) = a (In |r — ro| + qIn|r—rj| +c). (21.108) 
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Using plane polar coordinates, the radial (ie. normal) derivative of this function is given 
by 


aG 
GI ty) regs) 
op Ir| 
oF fate to gaan), 
2n\r| | |r—rol?  |r—1il? 


Using (21.107), on the perimeter of the circle p = a the radial derivative takes the form 
dG(r,to)| fr? = 9 - to glt|* — g(a? /|ro|?)r - to 
Op ~ 2nir| | Ir — tr)? (a /|rol?)t — ro? | 
1 
~ Ina |r — rol? 


p=a 


[ln]? + gltol? — (1+ g)r- ro], 
where we have set |r|? = a? in the second term on the RHS, but not in the first. If we take 
q = 1, the radial derivative simplifies to 
OG(r, ro) 
op 


or 1/L, where L is the circumference, and so (21.108) with q = 1 is the required Neumann 
Green’s function. 
Since p(r) = 0, the solution to our boundary-value problem is now given by (21.106) as 


ult) = (ute)}e— f Glesm)fte) dl, 
C 
where the integral is around the circumference of the circle C. In plane polar coordinates 


r= pcos ¢i+ psin gj and rp = pocos doi + po sin doj, and again using (21.107) we find 
that on C the Green’s function is given by 


1 
GF, to) |paa = > [nie Tol + In (us nl) + (| 


A 


= yen 
ie 2na 


2n rol 
1 2 a 
a In |r —ro|” + In +c 
2n |ro| 
ee {i [a’ + po — 2apo cos(@ — $o)| +inZ +eh. (21.109) 
2n Po 


Since dl = ad@ on C, the solution to the problem is given by 


2n 
u(po, $0) = (ude 5 J Hd) Inka? + 95 — apy cosld — dol a6. 


The contributions of the final two terms terms in the Green’s function (21.109) vanish 
because ie f(b) dd = 0. The average value of u around the circumference, (u)¢, is a freely 
specifiable constant as we would expect for a Neumann problem. This result should be 
compared with the result (21.104) for the corresponding Dirichlet problem, but it should 
be remembered that in the one case f(@) is a potential, and in the other the gradient of a 
potential. < 


21.6 Exercises 


21.1 Solve the following first-order partial differential equations by separating the 
variables: 
ou ou ou ou 
a) — —x— =0; b) x— -—2y— =0. 
(a) xO; (by x — Ve 
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21.4 


2155 


21.6 


A cube, made of material whose conductivity is k, has as its six faces the planes 
x = ta, y = ta and z = +a, and contains no internal heat sources. Verify that 
the temperature distribution 


RX 2 °HZ 2nn*t 
u(x, y,z,t) = Acos — sin — exp {| — 
a a 


a 


obeys the appropriate diffusion equation. Across which faces is there heat flow? 
What is the direction and rate of heat flow at the point (3a/4,a/4,a) at time 
t=a/(Kn’)? 

The wave equation describing the transverse vibrations of a stretched membrane 
under tension T and having a uniform surface density p is 


r oru : u\ Ou 

axt ap) Par 
Find a separable solution appropriate to a membrane stretched on a frame of 
length a and width b, showing that the natural angular frequencies of such a 


membrane are given by 
ee WT (Ww Fa mn? 
op Lae? bp? 


where n and m are any positive integers. 
Schrédinger’s equation for a non-relativistic particle in a constant potential region 
can be taken as 


(a) Find a solution, separable in the four independent variables, that can be 
written in the form of a plane wave, 
p(x, y, Zz, t) = Aexp[i(k -r— ot)]. 
Using the relationships associated with de Broglie (p = Ak) and Einstein 
(E = ha), show that the separation constants must be such that 
Pe +p; +p? =2mE. 


(b) Obtain a different separable solution describing a particle confined to a box 
of side a (y must vanish at the walls of the box). Show that the energy of 
the particle can only take the quantised values 

ir? 


fey EEE: Gin Dh phd 
E= aa (n, +n, +12), 


where nx, ny and n, are integers. 
Denoting the three terms of V* in spherical polars by V;, Vj, Vj, in an obvious 
way, evaluate V2u, etc. for the two functions given below and verify that, in each 


case, although the individual terms are not necessarily zero their sum V7u is zero. 
Identify the corresponding values of 7 and m. 


29 
fa) ulr.6,6) = (4P +5) SOP 


(b) u(r, 6,6) = (4" ie =) ini GetOtG: 


Prove that the expression given in equation (21.47) for the associated Legendre 
function P/"() satisfies the appropriate equation, (21.45), as follows. 
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21.8 


21.9 


21.10 


21.11 


(a) Evaluate dP}"(u)/du and d?P!"(u)/dp’, using the forms given in (21.47), and 
substitute them into (21.45). 

(b) Differentiate Legendre’s equation m times using Leibnitz’ theorem. 

(c) Show that the equations obtained in (a) and (b) are multiples of each other, 
and hence that the validity of (b) implies that of (a). 


Continue the analysis of exercise 10.20, concerned with the flow of a very viscous 
fluid past a sphere, to find the full expression for the stream function yp(r, 0). At 
the surface of the sphere r = a, the velocity field u = 0, whilst far from the sphere 
wp ~ (Ur? sin’ 6)/2. 

Show that f(r) can be expressed as a superposition of powers of r, and 
determine which powers give acceptable solutions. Hence show that 


3 
p(r, 8) (2 3ar + - ) sin’ 0. 


The motion of a very viscous fluid in the two-dimensional (wedge) region —« < 
& <a can be described, in (p, #) coordinates, by the (biharmonic) equation 


VV p = Vip =0, 


together with the boundary conditions 0w/0¢ = 0 at ¢ = +a, which represent 
the fact that there is no radial fluid velocity close to either of the bounding walls 
because of the viscosity, and 0w/dp = +p at @ = +a, which impose the condition 
that azimuthal flow increases linearly with r along any radial line. Assuming a 
solution in separated-variable form, show that the full expression for yp is 


p? sin2p —2¢cos 2a 
2 sin2«—2acos 2a 


p(p, 0) 


A circular disc of radius a is heated in such a way that its perimeter p = a has 
a steady temperature distribution A + Bcos’ ¢, where p and ¢ are plane polar 
coordinates and A and B are constants. Find the temperature T(p, @) everywhere 
in the region p <a. 

Consider possible solutions of Laplace’s equation inside a circular domain as 
follows. 


(a) Find the solution in plane polar coordinates p, ¢, that takes the value +1 
for 0 < @ < x and the value —1 for —x < ¢ < 0, when p =a. 

(b) For a point (x,y) on or inside the circle x? + y? = a’, identify the angles « 
and f defined by 


1 


a= tan and B =tan! : 
a+x a—x 
Show that u(x, y) = (2/z)(a + B) is a solution of Laplace’s equation that 
satisfies the boundary conditions given in (a). 
(c) Deduce a Fourier series expansion for the function 


sin sing 
tan7! ———"_. + tan"! ———_. 
1+cos¢ 1—cos¢ 
The free transverse vibrations of a thick rod satisfy the equation 
,ou Ou 
a= —_ = 
0x4 dt? 


Obtain a solution in separated-variable form and, for a rod clamped at one end, 
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21.16 


21.17 


x = 0, and free at the other, x = L, show that the angular frequency of vibration 


w satisfies 
o!?L o!?L 
cosh sec ; 
a a 


[At a clamped end both u and du/0x vanish, whilst at a free end, where there is 
no bending moment, 6?u/@x? and 63u/éx* are both zero. ] 

A membrane is stretched between two concentric rings of radii a and b (b > a). 
If the smaller ring is transversely distorted from the planar configuration by an 
amount c|¢|, —z < @ < 2, show that the membrane then has a shape given by 


cr In(b/p) 4c a” bn, 
u(p, p) 2 In(b/a) ie py, me(be" — em) \ pm p™ } cosmo. 


A string of length L, fixed at its two ends, is plucked at its mid-point by an 
amount A and then released. Prove that the subsequent displacement is given by 


oa) 


8A(—1)" . [(2n+1)ax (2n + 1)act 
u(x, t) = > Ont 1p sin a cos oe 


where, in the usual notation, c? = T/p. 
Find the total kinetic energy of the string when it passes through its unplucked 
position, by calculating it in each mode (each n) and summing, using the result 


3 1 _@ 
Qu ie = 78: 


Confirm that the total energy is equal to the work done in plucking the string 
initially. 

Prove that the potential for p < a associated with a vertical split cylinder of 
radius a, the two halves of which (cos¢ > 0 and cos < 0) are maintained at 
equal and opposite potentials +V, is given by 


_4V “ (—1)" spy 2m 
u(p, 6) = —— as oP (5) cos(2n + 1). 
A conducting spherical shell of radius a is cut round its equator and the two 
halves connected to voltages of +V and —V. Show that an expression for the 
potential at the point (r,0,¢) anywhere inside the two hemispheres is 


2 (—1)"(2n)M4n + 3) 72m 
u(r,0,@) =V ie ao [- Py 41(cos 8). 
2. PrInln+ 1! (a) 


[This is the spherical polar analogue of the previous question. ] 
A slice of biological material of thickness L is placed into a solution of a 
radioactive isotope of constant concentration Cp at time t = 0. For a later time t 
find the concentration of radioactive ions at a depth x inside one of its surfaces 
if the diffusion constant is x. 
Two identical copper bars are each of length a. Initially, one is at 0°C and the 
other is at 100°C; they are then joined together end to end and thermally isolated. 
Obtain in the form of a Fourier series an expression u(x,t) for the temperature 
at any point a distance x from the join at a later time t. Bear in mind the heat 
flow conditions at the free ends of the bars. 

Taking a = 0.5m estimate the time it takes for one of the free ends to 
attain a temperature of 55°C. The thermal conductivity of copper is 3.8 x 
10°Jm7!K7~'s—', and its specific heat capacity is 3.4 x 10°Jm~7K7!, 
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21.20 


21.21 


A sphere of radius a and thermal conductivity k; is surrounded by an infinite 
medium of conductivity k, in which far away the temperature tends to T,. 
A distribution of heat sources q(@) embedded in the sphere’s surface establish 
steady temperature fields T,(r,@) inside the sphere and T>(r, @) outside it. It can 
be shown, by considering the heat flow through a small volume that includes 
part of the sphere’s surface, that 


6 T; OT? - i 
Kia ie = q(@) on r=a. 
Given that 
1 fo.6) 
Ss nPn 0 A 
5 de InPr(cos 0) 


n=0 


find complete expressions for T,(r,0) and T>(r,0). What is the temperature at 
the centre of the sphere? 

Using result (21.74) from the worked example in the text, find the general 
expression for the temperature u(x,t) in the bar, given that the temperature 
distribution at time t = 0 is u(x,0) = exp(—x?/a’). 

Working in spherical polar coordinates r = (r,0,@), but for a system that has 
azimuthal symmetry around the polar axis, consider the following gravitational 
problem. 


(a) Show that the gravitational potential due to a uniform disc of radius a and 
mass M, centred at the origin, is given for r < a by 


2 4 
OM rl - “P, (cos 8) + = 5 (2) P3(cos 0) — z (=) P,(cos 0) +-- | F 
a 8 \a 


and for r > a by 
GM 
=a 


1-5 + (2) Patcosa) + : (£)' Pucose)— | : 


where the polar axis is normal to the plane of the disc. 

(b) Reconcile the presence of a term P,(cos@), which is odd under 0 > 2 — 0, 
with the symmetry with respect to the plane of the disc of the physical 
system. 

(c) Deduce that the gravitational field near an infinite sheet of matter of constant 
density p per unit area is 2xGp. 


In the region —co < x, y < «© and —t < z < t, a charge-density wave p(r) = 
Acos qx, in the x-direction, is represented by 


eld 0 ; 
plaje’™ da. 
els p(a) 


The resulting potential is ree by 


p(r) = 


Vir) = ” Pet da. 


al, 


Determine the relationship between V(a) and j(a), and hence show that the 
potential at the point (0,0,0) is 


A *  sinkt 
aa [. meg 
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Point charges gq and —qa/b (with a < b) are placed, respectively, at a point P, a 
distance b from the origin O, and a point Q between O and P, a distance a?/b 
from O. Show, by considering similar triangles QOS and SOP, where S is any 
point on the surface of the sphere centred at O and of radius a, that the net 
potential anywhere on the sphere due to the two charges is zero. 

Use this result (backed up by the uniqueness theorem) to find the force with 
which a point charge q placed a distance b from the centre of a spherical 
conductor of radius a (< b) is attracted to the sphere (i) if the sphere is earthed, 
and (ii) if the sphere is uncharged and insulated. 

Find the Green’s function G(r,ro) in the half-space z > 0 for the solution of 
V°® = 0 with ® specified in cylindrical polar coordinates (p,,z) on the plane 
z=0 by 


1 for p <1, 


O(e,0:2) = i for p> 1. 


Determine the variation of ®(0,0,z) along the z-axis. 

Electrostatic charge is distributed in a sphere of radius R centred on the origin. 
Determine the form of the resultant potential f(r) at distances much greater than 
R, as follows. 


(a) Express in the form of an integral over all space the solution of 


Veo, 


€0 


r-ri=r- "2 40(2). 
r r 


(c) Use results (a) and (b) to show that (r) has the form 
M d-r 1 
w= 24 St +-0(5). 
r r r 


Find expressions for M and d, and identify them physically. 


(b) Show that, for r >> 1’, 


Find, in the form of an infinite series, the Green’s function of the V* operator for 
the Dirichlet problem in the region -~7 <x <a, —-w <y<o, —c<z<c. 
Find the Green’s function for the three-dimensional Neumann problem 


Vp=0 forz>0 and —=f(x,y) onz=0. 


Determine (x, y,z) if 


f= ne for |x| <a, 


0 for |x| > a. 


Determine the Green’s function for the Klein—Gordon equation in a half-space 
as follows. 


(a) By applying the divergence theorem to the volume integral 


i [o(v? —r’)p —y(V? — mg dV, 
: 


obtain a Green’s function expression, as the sum of a volume integral and a 
surface integral, for the function ¢(r’) that satisfies 


Vp — wb =p 
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21.28 


21.1 
21.3 
21;5 


21.7 


21.9 

21.11 
21.13 
21.15 
21.17 


21.19 
21.21 
21.23 


in V and takes the specified form ¢ = f on S, the boundary of V. The 
Green’s function, G(r,r’), to be used satisfies 


V’G— WG = o(r—-r’) 


and vanishes when r is on S. 

(b) When V is all space, G(r,r’) can be written as G(t) = g(t)/t, where t = |r—r'| 
and g(t) is bounded as t > o. Find the form of G(t ‘a 

(c) Find @(r) in the half-space x > 0 if p(r) = d(r—r,) and ¢ = 0 both on x = 0 
and asr > 0. 


Consider the PDE £u(r) = p(r), for which the differential operator £ is given by 
L=V- [pV] + a(x), 


where p(r) and q(r) are functions of position. By proving the generalised form of 
Green’s theorem, 


| (¢Ly —wed)av = ¢ r(gVp — wV)- fds, 
Vv = 


show that the solution of the PDE is given by 


u(Fo) = | Gte.r0yo(ey aver) + fo (r) jw ue To) a ue) 


— G(r, rs dS(r), 


where G(r,ro) is the Green’s function satisfying G(r, ro) = d(r — ro). 


21.7 Hints and answers 


(a) C exp[A(x? + 2y)]5 (b) CO? y. 

u(x, y, 0) = sin(nax/a) sin(may/b)(A sin wt + B cos wt). 

(a) 6u/r?, —6u/r?, 0, 7 = 2 (or —3), m =0; 

(b) 2u/r?, (cot? 0 — 1)u/r?; —u/(r? sin? 0), 7 = 1 (or —2), m = +1. 

Solutions of the form r’ give / as —1,1,2,4. Because of the asymptotic form of 
yp, an r* term cannot be present. The coefficients of the three remaining terms are 
determined by the two boundary conditions u = 0 on the sphere and the form of 
p for large r. 

Express cos? ¢ in terms of cos2¢; T(p, d) = A + B/2 + (Bp*/2a’) cos 2¢. 
(Acosmx + B sinmx + C coshmx + D sinh mx) coer + ), with m‘at = @?. 

E, = 16pA?c?/[(2n + 1)°n?L]; E = 2pc?A?/L = aia (2Tv/(4L)] dv. 

Note that the boundary value function is a square wave aa is symmetric in ¢. 
Since there is no heat flow at x = +a, use a series of period 4a, u(x,0) = 100 for 
0 <x < 2a, u(x,0) = 0 for —2a< x <0. 


2004 1 . [(2n+1)nx k(2n + 1)?n?t 
u(x, t) = 50+ . Dea sn | | ex | : 


+1 2a 4a*s 


Taking i py n=0 term ae t = 2300 s. 

u(x,t) = [a/(a? + 4xt)'/?] exp[—x?/(@ + 4xt)]. 

Fourier-transform Poisson’s equation to show that p(a) = €0(a? + q?)V/(a). 
Follow the worked example that includes result (21.95). For part of the explicit 
integration, substitute p = z tana. 


al +22) — 24 (Le 21 


0(0,0,z) = Tae 
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DA2T: 


The terms in G(r,ro) that are additional to the fundamental solution are 


1 ioe) 
tn Sey { [(x — x0)? + (y — yo)? + (z + (-1)"z0 — ne)’ ] 


n=2 


-1/2 


+ [(x = x0)? + (y= yo)? + + (-1)"20 + ne] }. 


(a) As given in equation (21.86), but with ro replaced by r’. 
(b) Move the origin to r’ and integrate the defining Green’s equation to obtain 


t 
an — mt [ G(t’) 4nt’”? dt’ = 1, 
dt 0 
leading to G(t) = [—-1/(4zt)Je~™. 


(c) p(t) = [-1/(42) (pte — gq 'e-™), where p = |r—r,| and g = |r—r| with 
ry = (X1,91,21) and ry = (—X1, 1, 21): 
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Calculus of variations 


In chapters 2 and 5 we discussed how to find stationary values of functions of a 
single variable f(x), of several variables f(x, y,...) and of constrained variables, 
where x, y,... are subject to the n constraints gi(x,y,...) =0,i=1,2,...,n. In all 
these cases the forms of the functions f and g; were known, and the problem was 
one of finding the appropriate values of the variables x, y etc. 

We now turn to a different kind of problem in which we are interested in 
bringing about a particular condition for a given expression (usually maximising 
or minimising it) by varying the functions on which the expression depends. For 
instance, we might want to know in what shape a fixed length of rope should 
be arranged so as to enclose the largest possible area, or in what shape it will 
hang when suspended under gravity from two fixed points. In each case we are 
concerned with a general maximisation or minimisation criterion by which the 
function y(x) that satisfies the given problem may be found. 

The calculus of variations provides a method for finding the function y(x). 
The problem must first be expressed in a mathematical form, and the form 
most commonly applicable to such problems is an integral. In each of the above 
questions, the quantity that has to be maximised or minimised by an appropriate 
choice of the function y(x) may be expressed as an integral involving y(x) and 
the variables describing the geometry of the situation. 

In our example of the rope hanging from two fixed points, we need to find 
the shape function y(x) that minimises the gravitational potential energy of the 
rope. Each elementary piece of the rope has a gravitational potential energy 
proportional both to its vertical height above an arbitrary zero level and to the 
length of the piece. Therefore the total potential energy is given by an integral 
for the whole rope of such elementary contributions. The particular function y(x) 
for which the value of this integral is a minimum will give the shape assumed by 
the hanging rope. 
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CALCULUS OF VARIATIONS 


Figure 22.1 Possible paths for the integral (22.1). The solid line is the curve 
along which the integral is assumed stationary. The broken curves represent 
small variations from this path. 


So in general we are led by this type of question to study the value of an 
integral whose integrand has a specified form in terms of a certain function 
and its derivatives, and to study how that value changes when the form of 
the function is varied. Specifically, we aim to find the function that makes the 
integral stationary, i.e. the function that makes the value of the integral a local 
maximum or minimum. Note that, unless stated otherwise, y’ is used to denote 
dy/dx throughout this chapter. We also assume that all the functions we need to 
deal with are sufficiently smooth and differentiable. 


22.1 The Euler-Lagrange equation 


Let us consider the integral 


b 
I -| F(y,y',x) dx, (22.1) 


where a, b and the form of the function F are fixed by given considerations, 
e.g. the physics of the problem, but the curve y(x) is to be chosen so as to 
make stationary the value of J, which is clearly a function, or more accurately a 
functional, of this curve, i.e. I = I[y(x)]. Referring to figure 22.1, we wish to find 
the function y(x) (given, say, by the solid line) such that first-order small changes 
in it (for example the two broken lines) will make only second-order changes in 
the value of J. 

Writing this in a more mathematical form, let us suppose that y(x) is the 
function required to make J stationary and consider making the replacement 


Y(x) > y(x) + an (x), (22.2) 


where the parameter « is small and n(x) is an arbitrary function with sufficiently 
amenable mathematical properties. For the value of I to be stationary with respect 
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to these variations, we require 


dI 


rn =0 for all (x). (22.3) 


a=0 


Substituting (22.2) into (22.1) and expanding as a Taylor series in « we obtain 
b 
I(y, a) -| F(y + an, y! + on’, x) dx 
a 
b b 
OF OF 
= | Fly.y',x)dx =a + =a’ | dx + O(2’). 
i (y.y.x) x+ f Ga an x + O(a") 
With this form for I(y,«) the condition (22.3) implies that for all (x) we require 


b 
OF OF 
ol = — —n') dx =0 
[are ee 


where OJ denotes the first-order variation in the value of I due to the variation 
(22.2) in the function y(x). Integrating the second term by parts this becomes 


ar]? TAF od (OF 
bat l aoa (G)]mene ea 


In order to simplify the result we will assume, for the moment, that the end-points 
are fixed, i.e. not only a and b are given but also y(a) and y(b). This restriction 
means that we require 7(a) = y(b) = 0, in which case the first term on the LHS of 
(22.4) equals zero at both end-points. Since (22.4) must be satisfied for arbitrary 
n(x), it is easy to see that we require 

OF d (OF 

ae (=) (22.5) 

Oy dx \oy' 
This is known as the Euler-Lagrange (EL) equation, and is a differential equation 
for y(x), since the function F is known. 


22.2 Special cases 


In certain special cases a first integral of the EL equation can be obtained for a 
general form of F. 


22.2.1 F does not contain y explicitly 


In this case OF /dy = 0, and (22.5) can be integrated immediately giving 


OF 


ay = constant. (22.6) 
ay 
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B o(b, y(b)) 


ds 
dy 
dx 


A’ (a, y(a)) 


Figure 22.2. An arbitrary path between two fixed points. 


>» Show that the shortest curve joining two points is a straight line. 


Let the two points be labelled A and B and have coordinates (a, y(a)) and (b, y(b)) 
respectively (see figure 22.2). Whatever the shape of the curve joining A to B, the length 
of an element of path ds is given by 


ds = [(dx)? + (dy)"]"" = (1 + y")'dx, 


and hence the total path length along the curve is given by 


b 
L=f (1+ y?)!/2 dx. (22.7) 


We must now apply the results of the previous section to determine that path which makes 
L stationary (clearly a minimum in this case). Since the integral does not contain y (or 
indeed x) explicitly, we may use (22.6) to obtain 

OF y’ 


oy’ (1 + ye 
where k is a constant. This is easily rearranged and integrated to give 


k 


a— Keyra +6, 


y= 
which, as expected, is the equation of a straight line in the form y = mx +c, with 
m=k/(1—k?*)!/?. The value of m (or k) can be found by demanding that the straight line 
passes through the points A and B and is given by m = [y(b) — y(a)]/(b — a). Substituting 
the equation of the straight line into (22.7) we find that, again as expected, the total path 
length is given by 


L? = [y(b) — (a)? + (bay. < 
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ds a ‘aan 


r— dx 


Figure 22.3. A convex closed curve that is symmetrical about the x-axis. 


22.2.2 F does not contain x explicitly 


In this case, multiplying the EL equation (22.5) by y’ and using 
d { ,0F\ __,d (0F ia OF 
dx \Y ay) ax Vay J 7” 


jOF OF _ d (OF 
ey Y Gy! dx Y oy’ : 


we obtain 


But since F is a function of y and y’ only, and not explicitly of x, the LHS of 
this equation is just the total derivative of F, namely dF /dx. Hence, integrating 
we obtain 


F 
F—y— = constant. (22.8) 
y 


> Find the closed convex curve of length | that encloses the greatest possible area. 


Without any loss of generality we can assume that the curve passes through the origin 
and can further suppose that it is symmetric with respect to the x-axis; this assumption 
is not essential. Using the distance s along the curve, measured from the origin, as the 
independent variable and y as the dependent one, we have the boundary conditions 
y(0) = y(1/2) = 0. The element of area shown in figure 22.3 is then given by 


dA = ydx = y [(ds — (dy), 


and the total area by 
1/2 
A= 2f y(1 — y”)/2 ds: (22.9) 
0 


here y’ stands for dy/ds rather than dy/dx. Since the integrand does not contain s explicitly, 
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we can use (22.8) to obtain a first integral of the EL equation for y, namely 
y(l—y?)? + yyy"? =k, 
where k is a constant. On rearranging this gives 
ky! = 4 = y)!?, 
which, using y(0) = 0, integrates to 
y/k = sin(s/k). (22.10) 
The other end-point, y(//2) = 0, fixes the value of k as 1/(27) to yield 


From this we obtain dy = cos(2zs/1)ds and since (ds)? = (dx)? + (dy)? we find also that 
dx = +sin(2zs/1) ds. This in turn can be integrated and, using x(0) = 0, gives x in terms 
of s as 


aad oa Laas 
2n 0 ae 
Substituting the solution (22.10) into the expression for the total area (22.9), it is easily 


verified that A = [?/(4). A much quicker derivation of this result is possible using plane 
polar coordinates. < 


The previous two examples have been carried out in some detail, even though 
the answers are more easily obtained in other ways, expressly so that the method 
is transparent and the way in which it works can be filled in mentally at almost 
every step. The next example, however, does not have such an intuitively obvious 
solution. 


» Two rings, each of radius a, are placed parallel with their centres 2b apart and on a 
common normal. An open-ended axially symmetric soap film is formed between them (see 
figure 22.4). Find the shape assumed by the film. 


Creating the soap film requires an energy y per unit area (numerically equal to the surface 
tension of the soap solution). So the stable shape of the soap film, ie. the one that 
minimises the energy, will also be the one that minimises the surface area (neglecting 
gravitational effects). 

It is obvious that any convex surface, shaped such as that shown as the broken line in 
figure 22.4(a), cannot be a minimum but it is not clear whether some shape intermediate 
between the cylinder shown by solid lines in (a), with area 4zab (or twice this for the 
double surface of the film), and the form shown in (b), with area approximately 27a’, will 
produce a lower total area than both of these extremes. If there is such a shape (e.g. that in 
figure 22.4(c)), then it will be that which is the best compromise between two requirements, 
the need to minimise the ring-to-ring distance measured on the film surface (a) and the 
need to minimise the average waist measurement of the surface (b). 

We take cylindrical polar coordinates as in figure 22.4(c) and let the radius of the soap 
film at height z be p(z) with p(+b) = a. Counting only one side of the film, the element of 
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(a) (b) (c) 


Figure 22.4 Possible soap films between two parallel circular rings. 


surface area between z and z + dz is 


dS = 2np [(dz)? + (dp), 
so the total surface area is given by 
b 
S= an f pl + p”)"? dz. (22.11) 
—b 


Since the integrand does not contain z explicitly, we can use (22.8) to obtain an equation 
for p that minimises S, ie. 


(1+ p?)!? — pp"(1+ p?y? =k, 


where k is a constant. Multiplying through by (1+ pry, rearranging to find an explicit 


expression for p’ and integrating we find 


where c is the constant of integration. Using the boundary conditions p(+b) = a, we 
require c = 0 and k such that a/k = coshb/k (if b/a is too large, no such k can be found). 
Thus the curve that minimises the surface area is 


p/k = cosh(z/k), 


and in profile the soap film is a catenary (see section 22.4) with the minimum distance 
from the axis equal to k. < 


22.3 Some extensions 


It is quite possible to relax many of the restrictions we have imposed so far. For 
example, we can allow end-points that are constrained to lie on given curves rather 
than being fixed, or we can consider problems with several dependent and/or 
independent variables or higher-order derivatives of the dependent variable. Each 
of these extensions is now discussed. 
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22.3.1 Several dependent variables 


Here we have F = F(y1, yi,¥2,9)--+»¥ns nx) Where each y; = y;(x). The analysis 
in this case proceeds as before, leading to n separate but simultaneous equations 


for the y;(x), 
aF od (6F 
mn-aElm eee (22.12) 


oy; dx \ dy; 


22.3.2 Several independent variables 


With n independent variables, we need to extremise multiple integrals of the form 


oy oO 0 
=f [fre shee ee) Dedede: 
OX, OX2 OXn 


Using the same kind of analysis as before, we find that the extremising function 
y = y(X1,X2,...,Xn) must satisfy 


n 


OF 
rs 2, = (=), (22.13) 


where yx, stands for dy/0X;. 


22.3.3 Higher-order derivatives 


If in (22.1) F = F(y,y’,y”,...,y™,x) then using the same method as before 
and performing repeated integration by parts, it can be shown that the required 
extremising function y(x) satisfies 


OF d (OF a ( 0F a" ( OF 
sce Papacy (peas —~ | —) —...+(-1)"— {| —_) =0 22.14 
dy dx (=) + i (=) + OM be (ar) Oe 


a n—1) 


provided that y = y’ =--- = y! 0 at both end-points. If y, or any of its 
derivatives, is not zero at the end-points then a corresponding contribution or 
contributions will appear on the RHS of (22.14). 


22.3.4 Variable end-points 


We now discuss the very important generalisation to variable end-points. Suppose, 
as before, we wish to find the function y(x) that extremises the integral 


b 
r= | F(y, y',x) dx, 


but this time we demand only that the lower end-point is fixed, while we allow 
y(b) to be arbitrary. Repeating the analysis of section 22.1, we find from (22.4) 
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y(x) +(x) 
y(x) 
h(x, y) =0 eos 


Figure 22.5 Variation of the end-point b along the curve h(x, y) = 0. 


that we require 


b bra 
OF OF d (OF 
— —-—(— Ix = 0. 22.15 
Cralee E dx (57) ]ncoe oa 
Obviously the EL equation (22.5) must still hold for the second term on the LHS 
to vanish. Also, since the lower end-point is fixed, i.e. 7(a) = 0, the first term on 
the LHS automatically vanishes at the lower limit. However, in order that it also 
vanishes at the upper limit, we require in addition that 
OF 
ay! 


=) (22.16) 
x=b 


Clearly if both end-points may vary then 0F/dy’ must vanish at both ends. 

An interesting and more general case is where the lower end-point is again 
fixed at x = a, but the upper end-point is free to lie anywhere on the curve 
h(x, y) = 0. Now in this case, the variation in the value of IJ due to the arbitrary 
variation (22.2) is given to first order by 


b bra A 
oI = sor +f (S- as) n dx + F(b)Ax, (22.17) 
where Ax is the displacement in the x-direction of the upper end-point, as 
indicated in figure 22.5, and F(b) is the value of F at x = b. In order for (22.17) 
to be valid, we of course require the displacement Ax to be small. 
From the figure we see that Ay = n(b) + y’(b)Ax. Since the upper end-point 
must lie on h(x, y) = 0 we also require that, at x = 5, 


oh oh 
which on substituting our expression for Ay and rearranging becomes 


dh, oh dh 
(5 by )ax+ Sn =0. (22.18) 


x oy 
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y 


Figure 22.6 A frictionless wire along which a small bead slides. We seek the 
shape of the wire that allows the bead to travel from the origin O to the line 
xX = Xo in the least possible time. 


Now, from (22.17) the condition 6J = 0 requires, besides the EL equation, that 
at x = b, the other two contributions cancel, i.e. 
OF 


oy’ 
Eliminating Ax and 4 between (22.18) and (22.19) leads to the condition that at 
h OF Gh _ 


the end-point 
F ,OF \ 0 
Y ay} By Gy’ dx 


In the special case where the end-point is free to lie anywhere on the vertical line 
x = b, we have 0h/0x = 1 and 0h/dy = 0. Substituting these values into (22.20), 
we recover the end-point condition given in (22.16). 


FAx + —n =0. (22.19) 


(22.20) 


>A frictionless wire in a vertical plane connects two points A and B, A being higher than B. 
Let the position of A be fixed at the origin of an xy-coordinate system, but allow B to lie 


anywhere on the vertical line x = xo (see figure 22.6). Find the shape of the wire such that 
a bead placed on it at A will slide under gravity to B in the shortest possible time. 


This is a variant of the famous brachistochrone (shortest time) problem, which is often 
used to illustrate the calculus of variations. Conservation of energy tells us that the particle 
speed is given by 

v= 6 = V 2gy, 
where s is the path length along the wire and g is the acceleration due to gravity. Since 
the element of path length is ds = (1 + y"”)'/2dx, the total time taken to travel to the line 


X = Xo Is given by 
X=X0 dg 1 Xo 1 2 
t= i: ie | 4] ae dx. 
x=0 OU 28 Jo y 


Because the integrand does not contain x explicitly, we can use (22.8) with the specific 
form F = /1+ y?/7 to find a first integral; on simplification this yields 


1/2 


[pata] =k 
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where k is a constant. Letting a = k? and solving for y’ we find 


,_ dy _ fa-y 


sor me 


which on substituting y = asin’ 0 integrates to give 
x= $(20 —sin20) +c. 


Thus the parametric equations of the curve are given by 
x =b(¢@—singd) +c, y =b(1—cos@), 


where b = a/2 and ¢ = 26; they define a cycloid, the curve traced out by a point on 
the rim of a wheel of radius b rolling along the x-axis. We must now use the end-point 
conditions to determine the constants b and c. Since the curve passes through the origin, 
we see immediately that c = 0. Now since (xo) is arbitrary, i.e. the upper end-point can 
lie anywhere on the curve x = Xo, the condition (22.20) reduces to (22.16), so that we also 
require 

OF y’ 


OY’ | x=xg ya +y”) 


=0, 


X=X0 


which implies that y’ = 0 at x = xo. In words, the tangent to the cycloid at B must be 
parallel to the x-axis; this requires 7b = xo. < 


22.4 Constrained variation 


Just as the problem of finding thestationary values of a function f(x, y) subject to 
the constraint g(x, y) = constant is solved by means of Lagrange’s undetermined 
multipliers (see chapter 5), so the corresponding problem in the calculus of 
variations is solved by an analogous method. 

Suppose that we wish to find the stationary values of 


b 
T= | F(y, y', x) dx, 
a 
subject to the constraint that the value of 
b 
J= ‘ Gly, y', x) dx 
a 


is held constant. Following the method of Lagrange undetermined multipliers let 
us define a new functional 


b 
K=1+i= | (F + AG) dx, 


and find its unconstrained stationary values. Repeating the analysis of section 22.1 
we find that we require 


oF 4 (aF\, [9G 4 (a6)) _y 
dy dx \éy' 6y dx \ dy’ : 
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Figure 22.7. A uniform rope with fixed end-points suspended under gravity. 


which, together with the original constraint J = constant, will yield the required 
solution y(x). 

This method is easily generalised to cases with more than one constraint by the 
introduction of more Lagrange multipliers. If we wish to find the stationary values 
of an integral I subject to the multiple constraints that the values of the integrals 


J; be held constant for i = 1,2,...,n, then we simply find the unconstrained 
stationary values of the new integral 
n 
K=I+ a didi. 
1 


> Find the shape assumed by a uniform rope when suspended by its ends from two points 
at equal heights. 


We will solve this problem using x (see figure 22.7) as the independent variable. Let 
the rope of length 2L be suspended between the points x = +a, y = 0 (L > a) and 
have uniform linear density p. We then need to find the stationary value of the rope’s 
gravitational potential energy, 


t=—ps f yds=—pe f y(l-+ ya, 


with respect to small changes in the form of the rope but subject to the constraint that 
the total length of the rope remains constant, i.e. 


j= f as= | (1+ y?)!/2 dx = 2L. 


We thus define a new integral (omitting the factor —1 from I for brevity) 


K=1+i= f (pay + al + yds 


a 
and find its stationary values. Since the integrand does not contain the independent 
variable x explicitly, we can use (22.8) to find the first integral: 


1/2 


(pgy +4) (1 + y?) (pgy +A) (1 + yy yr =k, 
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where k is a constant; this reduces to 
eo) 
n (4 aE *) 
y= ar a 1. 


Making the substitution pgy + 2 = kcoshz, this can be integrated easily to give 
k p+A 
Joni (e**) res 
pg k 


where c is the constant of integration. 
We now have three unknowns, 4, k and c, that must be evaluated using the two end 
conditions y(+a) = 0 and the constraint J = 2L. The end conditions give 
cos PREF) = 4 = cosh = 
and since a # 0, these imply c = 0 and 4/k = cosh(pga/k). Putting c = 0 into the 
constraint, in which y’ = sinh(pgx/k), we obtain 


pee [ [1 + sinh? (28) dx 


= aan (288), 


Collecting together the values for the constants, the form adopted by the rope is therefore 


y(x) = * [cosh (=) cosh (=) “ 


where k is the solution of sinh(pga/k) = pgL/k. This curve is known as a catenary. < 


22.5 Physical variational principles 


Many results in both classical and quantum physics can be expressed as varia- 
tional principles, and it is often when expressed in this form that their physical 
meaning is most clearly understood. Moreover, once a physical phenomenon has 
been written as a variational principle, we can use all the results derived in this 
chapter to investigate its behaviour. It is usually possible to identify conserved 
quantities, or symmetries of the system of interest, that otherwise might be found 
only with considerable effort. From the wide range of physical variational princi- 
ples we will select two examples from familiar areas of classical physics, namely 
geometric optics and mechanics. 


22.5.1 Fermat’s principle in optics 


Fermat’s principle in geometrical optics states that a ray of light travelling in a 
region of variable refractive index follows a path such that the total optical path 
length (physical length x refractive index) is stationary. 
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95 


nz 


ny 


A 


Figure 22.8 Path of a light ray at the plane interface between media with 
refractive indices n; and n>, where ny < ny. 


>From Fermat’s principle deduce Snell’s law of refraction at an interface. 


Let the interface be at y = constant (see figure 22.8) and let it separate two regions with 
refractive indices n, and nz respectively. On a ray the element of physical path length is 
ds = (1+ y”)!/2dx, and so for a ray that passes through the points A and B, the total 
optical path length is 


B 
a / n(y)(1 + y?)!/? dx. 
A 


Since the integrand does not contain the independent variable x explicitly, we use (22.8) 
to obtain a first integral, which, after some rearrangement, reads 


n(y) (1+ y") 


where k is a constant. Recalling that y’ is the tangent of the angle @ between the 
instantaneous direction of the ray and the x-axis, this general result, which is not dependent 
on the configuration presently under consideration, can be put in the form 


=1/2 
= k, 


ncos @ = constant 


along a ray, even though n and ¢ vary individually. 

For our particular configuration n is constant in each medium and therefore so is 
y’. Thus the rays travel in straight lines in each medium (as anticipated in figure 22.8, 
but not assumed in our analysis), and since k is constant along the whole path we have 
n cos d; = 2 COs go, or in terms of the conventional angles in the figure 


n, sin 0, = nz sin 02. <4 


22.5.2 Hamilton’s principle in mechanics 


Consider a mechanical system whose configuration can be uniquely defined by a 
number of coordinates q; (usually distances and angles) together with time t and 
which experiences only forces derivable from a potential. Hamilton’s principle 
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dx Il) x 


Figure 22.9 Transverse displacement on a taut string that is fixed at two 
points a distance / apart. 


states that in moving from one configuration at time to to another at time t, the 
motion of such a system is such as to make 


th 
¢= / Handed ba (22.21) 
to 


stationary. The Lagrangian L is defined, in terms of the kinetic energy T and 
the potential energy V (with respect to some reference situation), by L = T —V. 
Here V is a function of the q; only, not of the g;. Applying the EL equation to £ 
we obtain Lagrange’s equations, 


Bh BE 
Ey (ees abe con, 
Oqi + (<=). ne Se 


> Using Hamilton's principle derive the wave equation for small transverse oscillations of a 
taut string. 


In this example we are in fact considering a generalisation of (22.21) to a case involving 
one isolated independent coordinate t, together with a continuum in which the q; become 
the continuous variable x. The expressions for T and V therefore become integrals over x 
rather than sums over the label i. 

If p and 7 are the local density and tension of the string, both of which may depend on 
x, then, referring to figure 22.9, the kinetic and potential energies of the string are given 


by 
1 a) 1 2 
a) 2 (2 af oe 
r= [ 5(2) dx, v= {3(2) dx 
and (22.21) becomes 
1 ! ay\? ay\? 
c=5f af |, (3) -(2) dx. 
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Using (22.13) and the fact that y does not appear explicitly, we obtain 


0 oy 0 oy 
a ce) oe (tax) 


If, in addition, p and t do not depend on x or t then 

ey 1 @y 

ax? 2 AP?” 
where c* = t/p. This is the wave equation for small transverse oscillations of a taut 
uniform string. < 


2 


22.6 General eigenvalue problems 


We have seen in this chapter that the problem of finding a curve that makes the 
value of a given integral stationary when the integral is taken along the curve 
results, in each case, in a differential equation for the curve. It is not a great 
extension to ask whether this may be used to solve differential equations, by 
setting up a suitable variational problem and then seeking ways other than the 
Euler equation of finding or estimating stationary solutions. 

We shall be concerned with differential equations of the form Ly = Ap(x)y, 
where the differential operator L is self-adjoint, so that £ = £* (with appropriate 
boundary conditions on the solution y) and p(x) is some weight function, as 
discussed in chapter 17. In particular, we will concentrate on the Sturm—Liouville 
equation as an explicit example, but much of what follows can be applied to 
other equations of this type. 

We have already discussed the solution of equations of the Sturm—Liouville 
type in chapter 17 and the same notation will be used here. In this section, 
however, we will adopt a variational approach to estimating the eigenvalues of 
such equations. 

Suppose we search for stationary values of the integral 


b 
r= f [poor aey*e0] ax (22.22) 


with y(a) = y(b) = 0 and p and q any sufficiently smooth and differentiable 
functions of x. However, in addition we impose a normalisation condition 


b 
J= / p(x)y?(x) dx = constant. (22.23) 


Here p(x) is a positive weight function defined in the interval a < x < b, but 
which may in particular cases be a constant. 
Then, as in section 22.4, we use undetermined Lagrange multipliers’ and 


S We use —A, rather than 2, so that the final equation (22.24) appears in the conventional Sturm— 
Liouville form. 
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consider K = I — /J given by 


b 
K= | [py —(q+Ap)y?| dx. 


On application of the EL equation (22.5) this yields 


a (=) +qy +dpy =0, (22.24) 
dx \" dx 

which is exactly the Sturm—Liouville equation (17.34), with eigenvalue 2. Now, 
since both IJ and J are quadratic in y and its derivative, finding stationary values 
of K is equivalent to finding stationary values of I/J. This may also be shown 
by considering the functional A = I/J, for which 


5A = (61/J) —(L/J*) 6 
= (61 — AdJ)/J 
= 6K/J. 


Hence, extremising A is equivalent to extremising K. Thus we have the important 
result that finding functions y that make I/J stationary is equivalent to finding 
functions y that are solutions of the Sturm—Liouville equation; the resulting value 
of I/J equals the corresponding eigenvalue of the equation. 

Of course this does not tell us how to find such a function y and, naturally, to 
have to do this by solving (22.24) directly defeats the purpose of the exercise. We 
will see in the next section how some progress can be made. It is worth recalling 
that the functions p(x), q(x) and p(x) can have many different forms, and so 
(22.24) represents quite a wide variety of equations. 

We now recall some properties of the solutions of the Sturm—Liouville equation. 
The eigenvalues 2; of (22.24) are real and will be assumed non-degenerate (for 
simplicity). We also assume that the corresponding eigenfunctions have been made 
real, so that normalised eigenfunctions y;(x) satisfy the orthogonality relation (as 
in (17.24)) 


b 
ViVjP dx = Oij. (22.25) 


Further, we take the boundary condition in the form 


x=b 
=0; (22.26) 


x=a 


[ivy] 


this can be satisfied by y(a) = y(b) = 0, but also by many other sets of boundary 
conditions. 
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> Show that 


b 
ib (yipy; — yiayi) dx = Abi. 


Let y; be an eigenfunction of (22.24), corresponding to a particular eigenvalue /;, so that 


(pyt)' + (q + Aip)yi = 0. 
Multiplying this through by y; and integrating from a to b (the first term by parts) we 
obtain 
b b b 
= 1 yi(pyi) dx + i) yi(q + Aip)yi dx = 0. (22.28) 


a a a 


E (ov) 


The first term vanishes by virtue of (22.26), and on rearranging the other terms and using 
(22.25), we find the result (22.27). < 


We see at once that, if the function y(x) minimises I /J, i.e. satisfies the Sturm— 
Liouville equation, then putting y; = y; = y in (22.25) and (22.27) yields J and 
I respectively on the left-hand sides; thus, as mentioned above, the minimised 
value of I/J is just the eigenvalue 4, introduced originally as the undetermined 
multiplier. 


>For a function y satisfying the Sturm—Liouville equation verify that, provided (22.26) is 
satisfied, 4 =I/J. 


Firstly, we multiply (22.24) through by y to give 
y(py’y + ay" + Apy? =0. 


Now integrating this expression by parts we have 


b b 7 b 
»ny| - / (py' — ay’) dx +4 i py dx = 0. 


The first term on the LHS is zero, the second is simply —J and the third is 2J. Thus 
A=I/J.<4 


22.7 Estimation of eigenvalues and eigenfunctions 


Since the eigenvalues 4; of the Sturm—Liouville equation are the stationary values 
of I/J (see above), it follows that any evaluation of I/J must yield a value that lies 
between the lowest and highest eigenvalues of the corresponding Sturm—Liouville 
equation, i.e. 


where, depending on the equation under consideration, either Amin = —oO and 
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Amax 18 finite, or Amax = 00 and Amin is finite. Notice that here we have departed 
from direct consideration of the minimising problem and made a statement about 
a calculation in which no actual minimisation is necessary. 

Thus, as an example, for an equation with a finite lowest eigenvalue J) any 
evaluation of I/J provides an upper bound on do. Further, we will now show that 
the estimate 12 obtained is a better estimate of Ao than the estimated (guessed) 
function y is of yo, the true eigenfunction corresponding to A. The sense in which 
‘better’ is used here will be clear from the final result. 

Firstly, we expand the estimated or trial function y in terms of the complete 
set yj: 


Y=Yotciyi t+coy2t+---, 


where, if a good trial function has been guessed, the c; will be small. Using (22.25) 
we have immediately that J = 1 + 7; |c;|*. The other required integral is 


b 2, 2 
[= if f (ss Py cai) —4q (» +>" au) | dx. 


On multiplying out the squared terms, all the cross terms vanish because of 
(22.27) to leave 


7 I 
Az 
J 
_ 4ot di leil7A; 
1+ » lel? 


=Agt+ SS lci?(Ai — 0) + O(c"). 


Hence 4 differs from 29 by a term second order in the c;, even though y differed 
from yo by a term first order in the c;; this is what we aimed to show. We notice 
incidentally that, since 29 < A; for all i, 2 is shown to be necessarily > Jo, with 
equality only if all c; = 0, ie. if y = yo. 

The method can be extended to the second and higher eigenvalues by imposing, 
in addition to the original constraints and boundary conditions, a restriction 
of the trial functions to only those that are orthogonal to the eigenfunctions 
corresponding to lower eigenvalues. (Of course, this requires complete or nearly 
complete knowledge of these latter eigenfunctions.) An example is given at the 
end of the chapter (exercise 22.25). 

We now illustrate the method we have discussed by considering a simple 
example, one for which, as on previous occasions, the answer is obvious. 
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y(x) 


0.85 


0.63 


0.44 


0.23 


Figure 22.10 Trial solutions used to estimate the lowest eigenvalue 2 of 
—y" = jy with y(0) = y'(1) = 0. They are: (a) y = sin(zx/2), the exact result; 
(b) y = 2x — x?; (c) y = x3 — 3x? + 3x; (d) y = sin?(ax/2). 


> Estimate the lowest eigenvalue of the equation 


ay 
es ae <= <a ‘ 
D2 Ay, 0<x<l, (22.29) 
with boundary conditions 
y(0) =0, y(1) =0. (22.30) 


We need to find the lowest value 29 of 4 for which (22.29) has a solution (x) that satisfies 
(22.30). The exact answer is of course y = Asin(xm/2) and Ay = 27/4 & 2.47. 

Firstly we note that the Sturm—Liouville equation reduces to (22.29) if we take p(x) = 1, 
q(x) = 0 and p(x) = 1 and that the boundary conditions satisfy (22.26). Thus we are able 
to apply the previous theory. 

We will use three trial functions so that the effect on the estimate of 4) of making better 
or worse ‘guesses’ can be seen. One further preliminary remark is relevant, namely that the 
estimate is independent of any constant multiplicative factor in the function used. This 
is easily verified by looking at the form of I/J. We normalise each trial function so that 
y(1) = 1, purely in order to facilitate comparison of the various function shapes. 

Figure 22.10 illustrates the trial functions used, curve (a) being the exact solution 
y = sin(zx/2). The other curves are (b) y(x) = 2x — x’, (c) y(x) = x° — 3x? + 3x, and (d) 
y(x) = sin?(1x/2). The choice of trial function is governed by the following considerations: 


(i) the boundary conditions (22.30) must be satisfied. 

(ii) a ‘good’ trial function ought to mimic the correct solution as far as possible, but 
it may not be easy to guess even the general shape of the correct solution in some 
cases. 

(iti) the evaluation of I/J should be as simple as possible. 
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It is easily verified that functions (b), (c) and (d) all satisfy (22.30) but, so far as mimicking 
the correct solution is concerned, we would expect from the figure that (b) would be 
superior to the other two. The three evaluations are straightforward, using (22.22) and 
(22.23): 
1 2 
ho bo@ 2x)? dx ms 4/3 250 
Jo (2x — x? P dx 8/15 


~ Io (3x2? = 6x+3)"dx 9/5 


é 


= = 2 9'80 
fo 08 — 3x2 +.3x)2dx 9/14 


fo (x2/4) sin?(nx) dx 12/8 


fy sin’ (nx/2) dx 3/8 


3.29. 


We expected all evaluations to yield estimates greater than the lowest eigenvalue, 2.47, 
and this is indeed so. From these trials alone we are able to say (only) that Ap < 2.50. 
As expected, the best approximation (b) to the true eigenfunction yields the lowest, and 
therefore the best, upper bound on Ao. < 


We may generalise the work of this section to other differential equations of 
the form Ly = Apy, where L = L’. In particular, one finds 


I 
Amin < 7 S Amaxs 
where J and J are now given by 
b b 
I -| y(Ly)dx and J -| py’ydx. (22.31) 
a a 


It is straightforward to show that, for the special case of the Sturm—Liouville 
equation, for which 


Ly = —(py') — ay, 
the expression for I in (22.31) leads to (22.22). 


22.8 Adjustment of parameters 


Instead of trying to estimate 4) by selecting a large number of different trial 
functions, we may also use trial functions that include one or more parameters 
which themselves may be adjusted to give the lowest value to 4 = I/J and 
hence the best estimate of 49. The justification for this method comes from the 
knowledge that no matter what form of function is chosen, nor what values are 
assigned to the parameters, provided the boundary conditions are satisfied 2 can 
never be less than the required Ap. 

To illustrate this method an example from quantum mechanics will be used. 
The time-independent Schrédinger equation is formally written as the eigenvalue 
equation Hy = Ey, where H is a linear operator, y the wavefunction describing 
a quantum mechanical system and E the energy of the system. The energy 
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operator H is called the Hamiltonian and for a particle of mass m moving in a 
one-dimensional harmonic oscillator potential is given by 


(22.32) 


where fi is Planck’s constant divided by 27. 


> Estimate the ground-state energy of a quantum harmonic oscillator. 


Using (22.32) in Hy = Ew, the Schrodinger equation is 


Pep kx? 
3 +S p=Eyp, —w<x<o. (22.33) 
The boundary conditions are that py should vanish as x + +oo0. Equation (22.33) is a form 
of the Sturm—Liouville equation in which p = h?/(2m), q kx?/2, p =1 and A= E; it 
can be solved by the methods developed previously, e.g. by writing the eigenfunction p as 
a power series in x. 

However, our purpose here is to illustrate variational methods and so we take as a trial 
wavefunction y = exp(—«x7), where « is a positive parameter whose value we will choose 
later. This function certainly > 0 as x > +o and is convenient for calculations. Whether 
it approximates the true wave function is unknown, but if it does not our estimate will 
still be valid, although the upper bound will be a poor one. 


With y = exp(—ax?) and therefore y’ = —2ax exp(—ax’), the required estimate is 
% [(h2/2m)4e2x2 + (k/2)x2Je"" dx hak 

pop ees a la (22.34) 

[oe dx 2m 8a 

This evaluation is easily carried out using the reduction formula 
n—1 ; ” —dyx2 
I,-2, for integrals of the form I, = ete aX. 

4a =a (22.35) 


So, we have obtained the estimate (22.34), involving the parameter «, for the oscillator’s 
ground-state energy, i.e. the lowest eigenvalue of H. In line with our previous discussion 
we now minimise 2 with respect to x. Putting dA/d« = 0 (clearly a minimum), yields 
a = (km)!/?/(2h), which in turn gives as the minimum value for 2 


A(k\'? ho 
ee (=) -%. (22.36) 


where we have put (k/m)'/? equal to the classical angular frequency . 

The method thus leads to the conclusion that the ground-state energy Eo is < sho. 
In fact, as is well known, the equality sign holds, sho being just the zero-point energy 
of a quantum mechanical oscillator. Our estimate gives the exact value because w(x) = 
exp(—ax?) is the correct functional form for the ground state wavefunction and the 
particular value of « that we have found is that needed to make p an eigenfunction of H 
with eigenvalue sho. < 


An alternative but equivalent approach to this is developed in the exercises 
that follow, as is an extension of this particular problem to estimating the second- 
lowest eigenvalue (see exercise 22.25). 
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22.1 


22.2 


22.3 


22.4 


22,5 


22.6 


22.7 


22.9 Exercises 


A surface of revolution, whose equation in cylindrical polar coordinates is p = 
p(z), is bounded by the circles p = a, z = +c (a > c). Show that the function 
that makes the surface integral J = fp? dS stationary with respect to small 
variations is given by p(z) =k + 27/(4k), where k = [a+ (a? — c?)!/7]/2. 

Show that the lowest value of the integral 


B y2y1/2 
gece. ine 
A y 


where A is (—1,1) and B is (1,1), is 2In(1+ /2). Assume that the Euler-Lagrange 
equation gives a minimising curve. 

The refractive index n of a medium is a function only of the distance r from a 
fixed point O. Prove that the equation of a light ray, assumed to lie in a plane 
through O, travelling in the medium satisfies (in plane polar coordinates) 


1 (dr \? _ Pwr) 1 
2(%) ~ Bama” 


where a is the distance of the ray from O at the point at which dr/dd = 0. 

If n = [1+(a?/r)]'/? and the ray starts and ends far from Q, find its deviation 
(the angle through which the ray is turned), if its minimum distance from O is a. 
The Lagrangian for a z-meson is given by 


L(x, 1) = 5(¢° — VO? — 1°9"), 


where 4 is the meson mass and ¢(x, ft) is its wavefunction. Assuming Hamilton’s 
principle, find the wave equation satisfied by ¢. 
Prove the following results about general systems. 


(a) For a system described in terms of coordinates q; and t, show that if t does 
not appear explicitly in the expressions for x, y and z (x = x(qj,t), etc.) then 
the kinetic energy T is a homogeneous quadratic function of the g; (it may 
also involve the q;). Deduce that 5°; gi(OT /0gi) = 2T. 

(b) Assuming that the forces acting on the system are derivable from a potential 
V, show, by expressing dT /dt in terms of q; and q;, that d(T + V)/dt = 


For a system specified by the coordinates q and t, show that the equation of 
motion is unchanged if the Lagrangian L(q, q,t) is replaced by 


aj: COD mat 


where ¢ is an arbitrary function. Deduce that the equation of motion of a particle 
that moves in one dimension subject to a force —dV(x)/dx (x being measured 
from a point O) is unchanged if O is forced to move with a constant velocity v 
(x still being measured from QO). 

In cylindrical polar coordinates, the curve (p(0),0,«p(@)) lies on the surface of 
the cone z = ap. Show that geodesics (curves of minimum length joining two 
points) on the cone satisfy 


p= [B p+ 07] 
where c is an arbitrary constant, but f has to have a particular value. Determine 
the form of (0) and hence find the equation of the shortest path on the cone 
between the points (R,—00,«R) and (R, 00, aR). 
[ You will find it useful to determine the form of the derivative of cos“! (u7'). ] 
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22.8 


22.9 


22.10 


22.11 


22.12 


22.13 


22.14 


22.15 


Derive the differential equations for the plane-polar coordinates, r and ¢, of a 
particle of unit mass moving in a field of potential V(r). Find the form of V if 
the path of the particle is given by r = asin ¢. 

You are provided with a line of length za/2 and negligible mass and some lead 
shot of total mass M. Use a variational method to determine how the lead shot 
must be distributed along the line if the loaded line is to hang in a circular arc 
of radius a when its ends are attached to two points at the same height. Measure 
the distance s along the line from its centre. 

Extend the result of subsection 22.2.2 to the case of several dependent variables 
yi(x), showing that, if x does not appear explicitly in the integrand, then a first 
integral of the Euler-Lagrange equations is 


n OF 
F- —— = constant. 
ay, ay 
A general result is that light travels through a variable medium by a path which 
minimises the travel time (this is an alternative formulation of Fermat’s principle). 
With respect to a particular cylindrical polar coordinate system (p, ¢, z), the speed 
of light v(p,@) is independent of z. If the path of the light is parameterised as 
p = p(z), d = G(z), use the result of the previous exercise to show that 
v(p + pp” +1) 

is constant along the path. 

For the particular case when v = v(p) = b(a* + p?)'/?, show that the two Euler— 
Lagrange equations have a common solution in which the light travels along a 
helical path given by ¢ = Az + B, p = C, provided that A has a particular value. 
Light travels in the vertical xz-plane through a slab of material which lies between 
the planes z = z and z = 2zo, and in which the speed of light v(z) = coz/Zo. 
Using the alternative formulation of Fermat’s principle, given in the previous 
question, show that the ray paths are arcs of circles. 

Deduce that, if a ray enters the material at (0,20) at an angle to the vertical, 

m/2—0, of more than 30°, then it does not reach the far side of the slab. 
A dam of capacity V (less than mb7h/2) is to be constructed on level ground next 
to a long straight wall which runs from (—b,0) to (b,0). This is to be achieved by 
joining the ends of a new wall, of height h, to those of the existing wall. Show 
that, in order to minimise the length L of new wall to be built, it should form 
part of a circle, and that L is then given by 


[. dx 
_p (l= 22x?)!/2” 


V sin} = (1 — 4p’)! 
hb? we a 


where / is found from 


and pu = Ab. 

In the brachistochrone problem of subsection 22.3.4 show that if the upper end- 
point can lie anywhere on the curve h(x, y) = 0, then the curve of quickest descent 
y(x) meets h(x, y) = 0 at right angles. 

The Schwarzchild metric for the static field of a non-rotating spherically sym- 
metric black hole of mass M is given by 


7 2GM (ary? 
(asp =e (: ee ) (at? — 7 —empen ~ 


Considering only motion confined to the plane 0 = 2/2, and assuming that the 


1? (do) —r’ sin? 6 (dd). 
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22.16 


22.17 


22.18 


22.19 


22.20 


22.21 


22.22 


path of a small test particle is such as to make f ds stationary, find two first 
integrals of the equations of motion. From their Newtonian limits, in which 
GM/r, 7? and r?¢° are all < c’, identify the constants of integration. 

Use result (22.27) to evaluate 


1 
J= / a — x?) P! (x)P/(x) dx, 


where P,,,(x) is a Legendre polynomial of order m. 
Determine the minimum value that the integral 


1 
p= [ ty"? 44rd 
0 


can have, given that y is not singular at x = 0 and that y(1) = y'(1) = 1. Assume 
that the Euler-Lagrange equation gives the lower limit, and verify retrospectively 
that your solution makes the first term on the LHS of equation (22.15) vanish. 
Show that y” — xy + Ax?y = 0 has a solution for which y(0) = y(1) = 0 and 
A < 147/4. 

Find an appropriate, but simple, trial function and use it to estimate the lowest 
eigenvalue 29 of Stokes’ equation, 


Te +Axy =0, with y(0) = y(z) = 0. 


Explain why your estimate must be strictly greater than /o. 
Estimate the lowest eigenvalue, Ao, of the equation 

ay 
ar ytdy=0, — y(-1)= (1) =0, 

x 

using a quadratic trial function. 
A drumskin is stretched across a fixed circular rim of radius a. Small transverse 
vibrations of the skin have an amplitude z(p, ¢,t) that satisfies 


1 é?z 
C2 ot? 


Vz= 


in plane polar coordinates. For a normal mode independent of azimuth, z = 
Z(p)coswt, find the differential equation satisfied by Z(p). By using a trial 
function of the form a” — p’, with adjustable parameter v, obtain an estimate for 
the lowest normal mode frequency. 
[The exact answer is (5.78)!/2c/a. ] 
Consider the problem of finding the lowest eigenvalue, Ao, of the equation 
ay dy 
(+ )55 42x 42y=0, y(t =0. 
(a) Recast the problem in variational form, and derive an approximation 2, to 
Ao by using the trial function y;(x) = 1 — x’. 
(b) Show that an improved estimate A, is obtained by using y2(x) = cos(zx/2). 
(c) Prove that the estimate A(y) obtained by taking y;(x) + yyo(x) as the trial 
function is 


_ 64/15 + 64y/m — 384y/n? + (7/3 + 1/2)y" 
- 16/15 + 64y/73 + 2 


Investigate A(y) numerically as y is varied, or, more simply, show that 
2(—1.80) = 3.668, an improvement on both 2; and A. 


Ay) 
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22.23 


22.24 


22.25 


22.26 


222] 


For the boundary conditions given below, obtain a functional A(y) whose sta- 
tionary values give the eigenvalues of the equation 


dy dy, ' 
pa + 2+xa + Ay = 0, y(0) =0, y'(2) =0. 


Derive an approximation to the lowest eigenvalue 2) using the trial function 
y(x) = xe~*/?. For what value(s) of y would 


(1 +x) 


y(x) = xe”? + B sin yx 


be a suitable trial function for attempting to obtain an improved estimate of A)? 
This is an alternative approach to the example in section 22.8. Using the notation 
of that section, the expectation value of the energy of the state y is given by 
{ w* Hyp dv. Denote the eigenfunctions of H by yj, so that Hyp; = E;y;, and, since 
H is self-adjoint (Hermitian), [ pypidv = dj. 


(a) By writing any function p as )> cj; and following an argument similar to 
that in section 22.7, show that 

_ fy Hyp do 

fw pdo 

the energy of the lowest state. This is the Rayleigh—Ritz principle. 


(b) Using the same trial function as in section 22.8, y = exp(—ax7), show that 
the same result is obtained. 


E 


0> 


This is an extension to section 22.8 and the previous question. With the ground- 
state (i.e. the lowest-energy) wavefunction as exp(—ax7), take as a trial function 
the orthogonal wave function x"*! exp(—ax?), using the integer n as a variable 
parameter. Use either Sturm—Liouville theory or the Rayleigh—Ritz principle to 
show that the energy of the second lowest state of a quantum harmonic oscillator 
is < 3h@/2. 

The Hamiltonian H for the hydrogen atom is 


Py = g 
2m 4neor 


For a spherically symmetric state, as may be assumed for the ground state, the 
only relevant part of V? is that involving differentiation with respect to r. 


ive) 
Jn Sy) re P dp 
0) 


and show that, for a trial wavefunction of the form exp(—fr) with 6 > 0, 
{Hy dv and f p*y dv (see exercise 22.24(a)) can be expressed as aJ, — bJ) 
and cJ> respectively, where a, b and c are factors which you should determine. 
Show that the estimate of E is minimised when B = mq?/(4zeoh’). 

Hence find an upper limit for the ground-state energy of the hydrogen atom. 
In fact, exp(—fr) is the correct form for the wavefunction and the limit gives 
the actual value. 


(a) Define the integrals J, by 


2s 


The upper and lower surfaces of a film of liquid, which has surface energy per 
unit area (surface tension) y and density p, have equations z = p(x) and z = q(x), 
respectively. The film has a given volume V (per unit depth in the y-direction) 
and lies in the region —L < x < L, with p(O) = q(0) = p(L) = q(L) = 0. The 
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22.28 


22.29 


22.1 
22.3 


total energy (per unit depth) of the film consists of its surface energy and its 
gravitational energy, and is expressed by 


L L 
B= 4% [ (—@dvty f [ato 404477] ax 


(a) Express V in terms of p and q. 
(b) Show that, if the total energy is minimised, p and q must satisfy 


12 y2 


P ee ee = constant. 


(l+p?y? (1+q")'? 
c Ss an approximate solution, consider the equations 
(c) A pproxi luti ider the equati 
p= a(L — |x|), q = W(L — |x\), 


where a and b are sufficiently small that a> and b* can be neglected compared 
with unity. Find the values of a and b that minimise E. 


A particle of mass m moves in a one-dimensional potential well of the form 


2,2 
V(x) = —p-~ sech Zax, 
m 


where y and « are positive constants. As in exercise 22.26, the expectation value 
(E) of the energy of the system is [ y*Hy dx, where the self-adjoint operator 
H is given by —(h?/2m)d?/dx? + V(x). Using trial wavefunctions of the form 
y = Asech fx, show the following: 


(a) for » = 1, there is an exact eigenfunction of H, with a corresponding (E) of 
half of the maximum depth of the well; 
(b) for u = 6, the ‘binding energy’ of the ground state is at least 10h?«?/(3m). 


[ You will find it useful to note that for u, v > 0, sech usech v > sech(u + v).] 
The Sturm—Liouville equation can be extended to two independent variables, x 
and z, with little modification. In equation (22.22), y’? is replaced by (Vy)? and 
the integrals of the various functions of y(x,z) become two-dimensional, ie. the 
infinitesimal is dx dz. 


The vibrations of a trampoline 4 units long and 1 unit wide satisfy the equation 
Vyt+ky=0. 


By taking the simplest possible permissible polynomial as a trial function, show 
that the lowest mode of vibration has k? < 10.63 and, by direct solution, that the 
actual value is 10.49. 


22.10 Hints and answers 


Note that the integrand, 2xp!/2(1 + p’)!/, does not contain z explicitly. 
I = f n(r)[r? + (dr/dd)’]' dd. Take axes such that ¢ = 0 when r = o. 
If 6 = (a — deviation angle)/2 then 6 = ¢ at r =a, and the equation reduces to 


B se dr 
(@ + 2)? a Mr aye? 


which can be evaluated by putting r = a(y + y!)/2, or successively r = acoshy, 
y =expy to yield a deviation of x[(a? + 07)'/? — a]/a. 
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22.5 


22.7 


22.9 


22.11 


22.13 


22.15 


22.17 


22.19 


22.21 


22.23 


22,25 


22:97 


22.29 


(a) 6x/ét = 0 and so x = D0, gi0x/0q;; (b) use 
_ d (0T oT 
DG (an) 7 # — Dotag, 


Use result (22.8); B? =1+4 7. 

Put p = uc to obtain d0/du = fB/[u(u? — 1)!/7]. Remember that cos! is a 
multivalued function; p(@) = [Rcos(00/f)]/[cos(0/f)]. 

—ly'(L— y?)-¥? = 2¢P(s), y = y(s), P(s) = fp p(s’) ds’. The solution, y 
Spesseley a 2P(na/4) = “Ml together give 2 = —gM. The required p(s) is 
given by [M/(2a)] sec?(s/a). 

Note that the @ E-L equation is automatically satisfied if v 4 v(). A = 1/a. 
Circle is #?x? + [Ay + (1 — 26°)? = 1. Use the fact that f ydx = V/h to 
determine the condition on /. 

Denoting (ds)*/(dt)? by f?, the Euler-Lagrange equation for ¢ gives r°@ = Af, 
where A corresponds to the angular momentum of the particle. Use the result 
of exercise 22.10 to obtain c? — (2GM/r) = Bf, where, to first order in small 
quantities, 


cB=c — cM + z +17°"), 
which reads ‘total energy = rest mass + gravitational energy + radial and 
azimuthal kinetic energy’. 
Convert the equation to the usual form, by writing y/(x) = u(x), and obtain 
x?u" + 4xu' — 4u = 0 with general solution Ax~* + Bx. Integrating a second time 
and using the boundary conditions gives y(x) = (1+ x7)/2 and J = 1; n(1) = 0, 
since y’(1) is fixed, and @F/éu' = 2x*u' =0 at x = 0. 
Using y = sinx as a trial function shows that 4) < 2/z. The estimate must be 
> Ao since the trial function does not satisfy the original equation. 
Z"+p'Z'+(w/c’Z =0, with Z(a) = 0 and Z'(0) = 0; this is an SL equation 
with p = p, q = 0 and weight function p/c?. Estimate of w* = [c?v/(2a7)] [0.5 — 
Av +2)-! +(2v +2)-"]-!, which minimises to c?(2 + ./2)?/(2a’) = 5.83c?/a* when 
y= 2. 
Note that the original equation is not self-adjoint; it needs an integrating factor 
of e°. A(y) = [fo 1 + xety? dxd/Lf, eXy? dx; 49 < 3/8. Since y/(2) must equal 0, 
y =(2/2)n+ 3) for some integer n. 
E, < (h@/2)(8n? + 12n + 3)/(4n + 1), which has a minimum value 3h@/2 when 
integer n = 0. 
(a) V= ies (p — q) dx. (c) Use V = (a— b)L? to eliminate b from the expression 
for E; now the minimisation is with respect to a alone. The values for a and b 
are +V/(2L*) — Vpg/(6y). 
The SL equation has p = 1, gq = 0, and p = 1. 
Use u(x,z) = x(4— x)z(1 — z) as a trial function; numerator = 1088/90, denomi- 
nator = 512/450. Direct solution k? = 1727/16. 
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Integral equations 


It is not unusual in the analysis of a physical system to encounter an equation 
in which an unknown but required function y(x), say, appears under an integral 
sign. Such an equation is called an integral equation, and in this chapter we discuss 
several methods for solving the more straightforward examples of such equations. 

Before embarking on our discussion of methods for solving various integral 
equations, we begin with a warning that many of the integral equations met in 
practice cannot be solved by the elementary methods presented here but must 
instead be solved numerically, usually on a computer. Nevertheless, the regular 
occurrence of several simple types of integral equation that may be solved 
analytically is sufficient reason to explore these equations more fully. 

We shall begin this chapter by discussing how a differential equation can be 
transformed into an integral equation and by considering the most common 
types of linear integral equation. After introducing the operator notation and 
considering the existence of solutions for various types of equation, we go on 
to discuss elementary methods of obtaining closed-form solutions of simple 
integral equations. We then consider the solution of integral equations in terms of 
infinite series and conclude by discussing the properties of integral equations with 
Hermitian kernels, i.e. those in which the integrands have particular symmetry 
properties. 


23.1 Obtaining an integral equation from a differential equation 


Integral equations occur in many situations, partly because we may always rewrite 
a differential equation as an integral equation. It is sometimes advantageous to 
make this transformation, since questions concerning the existence of a solu- 
tion are more easily answered for integral equations (see section 23.3), and, 
furthermore, an integral equation can incorporate automatically any boundary 
conditions on the solution. 
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We shall illustrate the principles involved by considering the differential equa- 
tion 
y"(x) = fy), (23.1) 


where f(x,y) can be any function of x and y but not of y’(x). Equation (23.1) 
thus represents a large class of linear and non-linear second-order differential 
equations. 

We can convert (23.1) into the corresponding integral equation by first inte- 
grating with respect to x to obtain 


yoy= [ feye)dz + or 
0 
Integrating once more, we find 
vex) = f au [ f(z. y(z))dz + ax + ©. 
0 0 


Provided we do not change the region in the uz-plane over which the double 
integral is taken, we can reverse the order of the two integrations. Changing the 
integration limits appropriately, we find 


y(x) = i “f(z, yz) dz / hit Abs Behe (23.2) 


= [afte vende + ox + ©; (23.3) 
0 


this is a non-linear (for general f(x, y)) Volterra integral equation. 

It is straightforward to incorporate any boundary conditions on the solution 
y(x) by fixing the constants c; and cz in (23.3). For example, we might have the 
one-point boundary condition y(0) = a and y’(0) = b, for which it is clear that 
we must set c} = b and c) =a. 


23.2 Types of integral equation 


From (23.3), we can see that even a relatively simple differential equation such 
as (23.1) can lead to a corresponding integral equation that is non-linear. In this 
chapter, however, we will restrict our attention to linear integral equations, which 
have the general form 


b 
g(x)y(x) = f(x) + af K(x, z)y(z) dz. (23.4) 


In (23.4), y(x) is the unknown function, while the functions f(x), g(x) and K(x, z) 
are assumed known. K(x,z) is called the kernel of the integral equation. The 
integration limits a and b are also assumed known, and may be constants or 
functions of x, and 4 is a known constant or parameter. 
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In fact, we shall be concerned with various special cases of (23.4), which are 
known by particular names. Firstly, if g(x) = 0 then the unknown function y(x) 
appears only under the integral sign, and (23.4) is called a linear integral equation 
of the first kind. Alternatively, if g(x) = 1, so that y(x) appears twice, once inside 
the integral and once outside, then (23.4) is called a linear integral equation of 
the second kind. In either case, if f(x) = 0 the equation is called homogeneous, 
otherwise inhomogeneous. 

We can distinguish further between different types of integral equation by the 
form of the integration limits a and b. If these limits are fixed constants then the 
equation is called a Fredholm equation. If, however, the upper limit b = x (ie. it 
is variable) then the equation is called a Volterra equation; such an equation is 
analogous to one with fixed limits but for which the kernel K(x,z) = 0 for z > x. 
Finally, we note that any equation for which either (or both) of the integration 
limits is infinite, or for which K(x, z) becomes infinite in the range of integration, 
is called a singular integral equation. 


23.3 Operator notation and the existence of solutions 


There is a close correspondence between linear integral equations and the matrix 
equations discussed in chapter 8. However, the former involve linear, integral rela- 
tions between functions in an infinite-dimensional function space (see chapter 17), 
whereas the latter specify linear relations among vectors in a finite-dimensional 
vector space. 

Since we are restricting our attention to linear integral equations, it will be 
convenient to introduce the linear integral operator K, whose action on an 
arbitrary function y is given by 


b 
ky= | K(x, z)y(z) dz. (23.5) 


This is analogous to the introduction in chapters 16 and 17 of the notation L to 
describe a linear differential operator. Furthermore, we may define the Hermitian 
conjugate K* by 


b 
Kty= f K"(z,x)y(z) dz, 


where the asterisk denotes complex conjugation and we have reversed the order 
of the arguments in the kernel. 

It is clear from (23.5) that K is indeed linear. Moreover, since K operates on 
the infinite-dimensional space of (reasonable) functions, we may make an obvious 
analogy with matrix equations and consider the action of K on a function f as 
that of a matrix on a column vector (both of infinite dimension). 

When written in operator form, the integral equations discussed in the pre- 
vious section resemble equations familiar from linear algebra. For example, the 
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inhomogeneous Fredholm equation of the first kind may be written as 
O=f +AKy, 


which has the unique solution y = —K~'f/A, provided that f # 0 and the inverse 
operator K~! exists. 

Similarly, we may write the corresponding Fredholm equation of the second 
kind as 


y=f tAKy. (23.6) 


In the homogeneous case, where f = 0, this reduces to y = AK y, which is 
reminiscent of an eigenvalue problem in linear algebra (except that 1 appears on 
the other side of the equation) and, similarly, only has solutions for at most a 
countably infinite set of eigenvalues 4;. The corresponding solutions y; are called 
the eigenfunctions. 

In the inhomogeneous case (f # 0), the solution to (23.6) can be written 
symbolically as 


y=(1—JKy'f, 


again provided that the inverse operator exists. It may be shown that, in general, 
(23.6) does possess a unique solution if 2 # Jj, ie. when 2 does not equal one of 
the eigenvalues of the corresponding homogeneous equation. 

When 4 does equal one of these eigenvalues, (23.6) may have either many 
solutions or no solution at all, depending on the form of f. If the function f is 
orthogonal to every eigenfunction of the equation 


g=iUKig (23.7) 


that belongs to the eigenvalue 2", ie. 


b 
wif) = | s'corerdx=0 


for every function g obeying (23.7), then it can be shown that (23.6) has many 
solutions. Otherwise the equation has no solution. These statements are discussed 
further in section 23.7, for the special case of integral equations with Hermitian 
kernels, i.e. those for which K = Ky’. 


23.4 Closed-form solutions 


In certain very special cases, it may be possible to obtain a closed-form solution 
of an integral equation. The reader should realise, however, when faced with an 
integral equation, that in general it will not be soluble by the simple methods 
presented in this section but must instead be solved using (numerical) iterative 
methods, such as those outlined in section 23.5. 
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23.4.1 Separable kernels 


The most straightforward integral equations to solve are Fredholm equations 
with separable (or degenerate) kernels. A kernel is separable if it has the form 


K(x,z) = >> di(x)pilz), (23.8) 
i=1 


where #;(x) are y;(z) are respectively functions of x only and of z only and the 
number of terms in the sum, n, is finite. 

Let us consider the solution of the (inhomogeneous) Fredholm equation of the 
second kind, 


b 
y(x) = f(x) + i | K(x, z)y(z) dz, (23.9) 


which has a separable kernel of the form (23.8). Writing the kernel in its separated 
form, the functions ¢;(x) may be taken outside the integral over z to obtain 


n b 
vl) =F) +290 dies) | yin) de. 
i=1 f 


Since the integration limits a and b are constant for a Fredholm equation, the 
integral over z in each term of the sum is just a constant. Denoting these constants 
by 


b 
a= f wi(z)y(z) dz, (23.10) 


the solution to (23.9) is found to be 


n 


V(x) = FO) FAD? cidi), (23.11) 


i=1 


where the constants c; can be evalutated by substituting (23.11) into (23.10). 


> Solve the integral equation 


1 
y(x) =x+ af (xz + 2*)y(z) dz. (23.12) 
0 


The kernel for this equation is K(x, z) = xz + z?, which is clearly separable, and using the 
notation in (23.8) we have $;(x) = x, b2(x) = 1, yi(z) = z and y(z) = z?. From (23.11) 
the solution to (23.12) has the form 

V(x) = x + Ae1x + €2), 


where the constants c; and c2 are given by (23.10) as 


1 

“g= Y z[z+Acz+c)]dz = ; + zAci + 4202, 
0 
1 

Q= | 2 [z +A(c1z +0)) dz = ; + faci + zAcp. 
0 
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These two simultaneous linear equations may be straightforwardly solved for c; and c2 to 
give 

7 2442 snd = 18 

== Sea 


so that the solution to (23.12) is 


Cy 


ve (72 — 24A)x + 182 

72 — 481 — 72 

In the above example, we see that (23.12) has a (finite) unique solution provided 
that A is not equal to either root of the quadratic in the denominator of y(x). 
The roots of this quadratic are in fact the eigenvalues of the corresponding 
homogeneous equation, as mentioned in the previous section. In general, if the 
separable kernel contains n terms, as in (23.8), there will be n such eigenvalues, 
although they may not all be different. 

Kernels consisting of trigonometric (or hyperbolic) functions of sums or differ- 
ences of x and z are also often separable. 


Find the eigenvalues and corresponding eigenfunctions of the homogeneous Fredholm 
equation 


y(x) = me sin(x + z) y(z) dz. (23.13) 


The kernel of this integral equation can be written in separated form as 
K(x, z) = sin(x + z) = sinx cosz + cos x sin z, 
so, comparing with (23.8), we have ¢,(x) = sinx, ¢2(x) = cosx, pi(z) = cosz and 
w2(z) = sinz. 
Thus, from (23.11), the solution to (23.13) has the form 


y(x) = A(cy sin x + cz cos x), 


where the constants c; and c are given by 


i a 
q= af cosz(c;sinz+c,cosz)dz = ee, (23.14) 
0 
Pa ae : An 
a= af sin z(c; sinz + c)cosz)dz = tt: (23.15) 
0 
Combining these two equations we find c; = (An/2)?c,, and, assuming that c,; # 0, this 


gives 2 = +2/z, the two eigenvalues of the integral equation (23.13). 

By substituting each of the eigenvalues back into (23.14) and (23.15), we find that 
the eigenfunctions corresponding to the eigenvalues 2; = 2/a and 4, = —2/z are given 
respectively by 

yi(x) = A(sin x + cos x) and yo(x) = B(sin x — cos x), (23.16) 


where A and B are arbitrary constants. < 
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23.4.2 Integral transform methods 


If the kernel of an integral equation can be written as a function of the difference 
x —z of its two arguments, then it is called a displacement kernel. An integral 
equation having such a kernel, and which also has the integration limits —co to 
oo, may be solved by the use of Fourier transforms (chapter 13). 

If we consider the following integral equation with a displacement kernel, 


ee) 


y(x) = f(x) +4 K(x —z)y(z)dz (23.17) 


—00 


the integral over z clearly takes the form of a convolution (see chapter 13). 
Therefore, Fourier-transforming (23.17) and using the convolution theorem, we 
obtain 


H(k) = F(k) + V20AK (kK), 
which may be rearranged to give 
art | i(k) 
Hk) = 1 ma (23.18) 


Taking the inverse Fourier transform, the solution to (23.17) is given by 


P(k)explikx) exp(ikx) dk 


Yn) = ei 1— PmiK 


If we can perform this inverse Fourier transformation then the solution can be 
found explicitly; otherwise it must be left in the form of an integral. 


> Find the Fourier transform of the function 


_ jl if|x|<a, 
Hor={ if |x| >a. 


Hence find an explicit expression for the solution of the integral equation 


° sin(x — z) 


v(x) = fo +2 fo 


Find the solution for the special case f(x) = (sinx)/x. 


y(z) dz. (23.19) 


The Fourier transform of g(x) is given directly by 


7” _1 exp(—ikx) ikx) = 25a 
&(k) = se [ec ikx) dx = Bes ib |. ee isn 


The kernel of the integral equation (23.19) is K(x — z) = [sin(x — z)]/(x — z). Using 
(23.20), it is straightforward to show that the Fourier transform of the kernel is 


si, _ J Va/2 if lkl <1, 
K(k) = . ret (23.21) 
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Thus, using (23.18), we find the Fourier transform of the solution to be 


ep — J(K)/A— nd) if |kl <1, 
Hk) = {iH if |k| > 1. (23:22) 


Inverse Fourier-transforming, and writing the result in a slightly more convenient form, 
the solution to (23.19) is given by 


1 ig 
v(x) = foo) + tr _ i) = i Fe) exp(ikx) dk 
= f(x)4 =f T( k) exp(ikx) dk. (23.23) 


Ton = Vin 
It is clear from (23.22) that when 2 = 1/z, which is the only eigenvalue of the 
corresponding homogeneous equation to (23.19), the solution becomes infinite, as we 
would expect. ‘ 
For the special case f(x) = (sin x)/x, the Fourier transform f(k) is identical to that in 
(23.21), and the solution (23.23) becomes 


, sin x k 
y(x) (Ae, [expt 
_ sinx TA exp(ikx) Ee 
ee (5); 2 ix ix 
sin x +( tA ) sin x ( 1 ) sin x > 
x 1-7 x 1-7 x 


If, instead, the integral equation (23.17) had integration limits 0 and x (so 
making it a Volterra equation) then its solution could be found, in a similar way, 


by using the convolution theorem for Laplace transforms (see chapter 13). We 
would find 
(5) = 20, 
1 —1K(s) 
where s is the Laplace transform variable. Often one may use the dictionary of 
Laplace transforms given in table 13.1 to invert this equation and find the solution 
y(x). In general, however, the evaluation of inverse Laplace transform integrals 
is difficult, since (in principle) it requires a contour integration; see chapter 24. 
As a final example of the use of Fourier transforms in solving integral equations, 
we mention equations that have integration limits —oo and oo and a kernel of the 
form 


K(x, z) = exp(—ixz). 


Consider, for example, the inhomogeneous Fredholm equation 


ee) 


y(x) = f(x) + af exp(—ixz) y(z) dz. (23.24) 


The integral over z is clearly just (a multiple of) the Fourier transform of y(z), 
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so we can write 
y(x) = f(x) + V2n AF). (23.25) 


If we now take the Fourier transform of (23.25) but continue to denote the 
independent variable by x (ie. rather than k, for example), we obtain 


H(x) = F(x) + V2nAy(—x). (23.26) 
Substituting (23.26) into (23.25) we find 
V(x) = flo) + Vind [Ft + Vixy(—v)], 
but on making the change x — —x and substituting back in for y(—x), this gives 
V(x) = f(x) + V2WAF (x) + 27? [f(—x) + V2RAF(—x) + 2ni?y(x)] 
Thus the solution to (23.24) is given by 


[F0) + 2m)" AF) + 2nd f(—x) + (2m)23F(—2)] 
(23.27) 


1 
YO) = Toa 


Clearly, (23.24) possesses a unique solution provided 24 # +1/,/2a or +i/,/27; 
these are easily shown to be the eigenvalues of the corresponding homogeneous 
equation (for which f(x) = 0). 


> Solve the integral equation 


vis) exp (=) a, ext-iney nerd, (23.28) 


0 


where 4 is areal constant. Show that the solution is unique unless 2 has one of two particular 
values. Does a solution exist for either of these two values of 2? 


Following the argument given above, the solution to (23.28) is given by (23.27) with 
f(x) = exp(—x?/2). In order to write the solution explicitly, however, we must calculate 
the Fourier transform of f(x). Using equation (13.7), we find f(k) = exp(—k?/2), from 
which we note that f(x) has the special property that its functional form is identical to 
that of its Fourier transform. Thus, the solution to (23.28) is given by 


_ 1 1/2) 2 3/253 x? 
WO) = Togas Lt Ce) A+ 2nd? + (2n)*723] exp ee 


(23.29) 


Since / is restricted to be real, the solution to (23.28) will be unique unless 4 = +1/./2z, 
at which points (23.29) becomes infinite. In order to find whether solutions exist for either 
of these values of 2 we must return to equations (23.25) and (23.26). 

Let us first consider the case 2 = +1/./2n. Putting this value into (23.25) and (23.26), 
we obtain 


y(x) x) + F(x), (23.30) 
3(x) x) + y(—x). (23.31) 


F( 
F( 


x 
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Substituting (23.31) into (23.30) we find 
yx) = f(x) + Ff) + ¥(—»), 
but on changing x to —x and substituting back in for y(—x), this gives 
yx) = F(0) + FO) + f(—») + F(-) + yO). 
Thus, in order for a solution to exist, we require that the function f(x) obeys 
f(x) + F(x) + f(—x) + F(x) = 0. 
This is satisfied if f(x) = —f(x), ic. if the functional form of f(x) is minus the form of its 
Fourier transform. We may repeat this analysis for the case 1 = —1/,/2z, and, in a similar 


way, we find that this time we require f(x) = f(x). bs 

In our case f(x) = exp(—x?/2), for which, as we mentioned above, f(x) = f(x). 
Therefore, (23.28) possesses no solution when 4 = +1/./2z but has many solutions when 
1 =—1//2n. < 

A similar approach to the above may be taken to solve equations with kernels 
of the form K(x, y) = cos xy or sin xy, either by considering the integral over y in 
each case as the real or imaginary part of the corresponding Fourier transform 
or by using Fourier cosine or sine transforms directly. 


23.4.3 Differentiation 


A closed-form solution to a Volterra equation may sometimes be obtained by 
differentiating the equation to obtain the corresponding differential equation, 
which may be easier to solve. 


> Solve the integral equation 


Dividing through by x, we obtain 
py et [ 2 y(z)dz, 
x 0 


which may be differentiated with respect to x to give 


d [ y(x) 5 y(x) 
[22] en [5 


dx | x 
This equation may be integrated straightforwardly, and we find 
; 4 
if | oe ee 
x 4 


where c is a constant of integration. Thus the solution to (23.32) has the form 


x4 
y(x) = Ax exp (-5) ‘ (23.33) 


where A is an arbitrary constant. 

Since the original integral equation (23.32) contains no arbitrary constants, neither 
should its solution. We may calculate the value of the constant, A, by substituting the 
solution (23.33) back into (23.32), from which we find A = 1. < 
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23.5 Neumann series 


As mentioned above, most integral equations met in practice will not be of the 

simple forms discussed in the last section and so, in general, it is not possible to 

find closed-form solutions. In such cases, we might try to obtain a solution in the 

form of an infinite series, as we did for differential equations (see chapter 16). 
Let us consider the equation 


b 
y(x) = f(x) + i | K(x, z)y(z) dz, (23.34) 


where either both integration limits are constants (for a Fredholm equation) or 
the upper limit is variable (for a Volterra equation). Clearly, if 2 were small then 
a crude (but reasonable) approximation to the solution would be 


V(x) © yo(x) = f(x), 


where yo(x) stands for our ‘zeroth-order’ approximation to the solution (and is 
not to be confused with an eigenfunction). 

Substituting this crude guess under the integral sign in the original equation, 
we obtain what should be a better approximation: 


b b 
yx) = fe) +a / K(x2)po(z) dz = f(x) +4 / K(x,2)f(2) dz, 


which is first order in 7. Repeating the procedure once more results in the 
second-order approximation 


b 
yo(x) = fo) +4 ] K(x,2)y1(2)dz 


b b b 
= f(x) +a f K(xziyfter)da +22 [ dz [ K(x, 21)K (21,22) f (z2) dz. 


It is clear that we may continue this process to obtain progressively higher-order 
approximations to the solution. Introducing the functions 


K(x, z) = K(x,z), 


b 
Kalx,z)= f K(x, 21)K (21, 2) dz1, 


b b 
K3(%z) = | dey i) K (20K (21, 25) (2,2) deo, 


and so on, which obey the recurrence relation 
b 
Kale) = fo K(62)Kyae12)d21, 
a 
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we may write the nth-order approximation as 


n b 
vals) = FC) +72" [ Kylasz)fle) dz (23.35) 


m=1 


The solution to the original integral equation is then given by y(x) = 
limy+« Vn(x), provided the infinite series converges. Using (23.35), this solution 
may be written as 


b 

v(x) = f(x) +4 if R(x, 25 Af (2) dz, (23.36) 
a 
where the resolvent kernel R(x,z;A) is given by 

R(x,z;4) = be A" K ma s(X Z)- (23.37) 

m=0 
Clearly, the resolvent kernel, and hence the series solution, will converge 
provided / is sufficiently small. In fact, it may be shown that the series converges 


in some domain of |A| provided the original kernel K(x,z) is bounded in such a 
way that 


b b 
ue f ax [ |K(x,z)/? dz < 1. (23.38) 


> Use the Neumann series method to solve the integral equation 


y(x)=x+ ae xzy(z) dz. 
0 


Following the method outlined above, we begin with the crude approximation y(x) ~ 
yo(x) = x. Substituting this under the integral sign in (23.39), we obtain the next approxi- 
mation 
1 1 ‘ 
2 Ax 
yi(x) =x4if XZyo(z) dz =x+if xz°dz = x+ > 
0 0 


Repeating the procedure once more, we obtain 


1 
yo(x) =x+ af xzyi(z) dz 


atifle pee dz = x+ A ike % 
Skee fe 3 Se ge gy 


For this simple example, it is easy to see that by continuing this process the solution to 


(23.39) is obtained as 
Ai (2 E i 2 " 
3 3 3 


Clearly the expression in brackets is an infinite geometric series with first term 2/3 and 


y(x) =x + 
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common ratio 4/3. Thus, provided |j| < 3, this infinite series converges to the value 
4/(3 — 2), and the solution to (23.39) is 


Ax 3x 
y(x) =x+ CS ee oe (23.40) 


Finally, we note that the requirement that |A| < 3 may also be derived very easily from 
the condition (23.38). < 


23.6 Fredholm theory 


In the previous section, we found that a solution to the integral equation (23.34) 
can be obtained as a Neumann series of the form (23.36), where the resolvent 
kernel R(x,z;2) is written as an infinite power series in 2. This solution is valid 
provided the infinite series converges. 

A related, but more elegant, approach to the solution of integral equations 
using infinite series was found by Fredholm. We will not reproduce Fredholm’s 
analysis here, but merely state the results we need. Essentially, Fredholm theory 
provides a formula for the resolvent kernel R(x,z;2) in (23.36) in terms of the 
ratio of two infinite series: 


yy D(x, 234) 
R(x, 232) = Tay * (23.41) 
The numerator and denominator in (23.41) are given by 
: wt" yn 
D(x,z34) = aI D,(x,z)A", (23.42) 
n=0 . 
« —1)" 
dij) = ( _ d,A", (23.43) 
n=0 : 


where the functions D,(x,z) and the constants d, are found from recurrence 
relations as follows. We start with 


Do(x, Zz) = K(x, Zz) and dy = 1, (23.44) 


where K(x,z) is the kernel of the original integral equation (23.34). The higher- 
order coefficients of 4 in (23.43) and (23.42) are then obtained from the two 
recurrence relations 


b 
n= f Dy_-1(x, x) dx, (23.45) 


b 
D,(x, Zz) = K(x,z)dn — nf K(x, 21)Dn-1(21, Z) dz1. (23.46) 


Although the formulae for the resolvent kernel appear complicated, they are 
often simple to apply. Moreover, for the Fredholm solution the power series 
(23.42) and (23.43) are both guaranteed to converge for all values of A, unlike 
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Neumann series, which converge only if the condition (23.38) is satisfied. Thus the 
Fredholm method leads to a unique, non-singular solution, provided that d(/) # 0. 
In fact, as we might suspect, the solutions of d(/) = 0 give the eigenvalues of the 
homogeneous equation corresponding to (23.34), Le. with f(x) = 0. 


> Use Fredholm theory to solve the integral equation (23.39). 


Using (23.36) and (23.41), the solution to (23.39) can be written in the form 
' D(x,z;2) 
d(A) 

In order to find the form of the resolvent kernel R(x,z;2), we begin by setting 


Do(x,z) = K(x,z) = xz and dg =1 


1 
y(x) =x+ af R(x,23A)zdz =x+ af zdz. (23.47) 
0 0 


and use the recurrence relations (23.45) and (23.46) to obtain 


1 1 
1 
a= Dotx.xydx = [ dx = =, 
0 0 3 
D,(x,z) = [tea Sas Zi = 
i a Tf a Bile 


Applying the recurrence relations again we find that d, = 0 and D,(x,z) = 0 for n > 1. 
Thus, from (23.42) and (23.43), the numerator and denominator of the resolvent respectively 
are given by 


D(x,z;4) = xz and d(a)=1-— 


Wl 


Substituting these expressions into (23.47), we find that the solution to (23.39) is given 
by 


eed 
x)=x+A4/ —-<d 
yoya sta f as 
3x 


ae x oy i AX 
” “11-4/3 3], °° 3-4 3-2’ 


which, as expected, is the same as the solution (23.40) found by constructing a Neumann 
series. < 


23.7 Schmidt—Hilbert theory 


The Schmidt-Hilbert (SH) theory of integral equations may be considered as 
analogous to the Sturm—Liouville (SL) theory of differential equations, discussed 
in chapter 17, and is concerned with the properties of integral equations with 
Hermitian kernels. An Hermitian kernel enjoys the property 


K(x,z) = K*(z,x), (23.48) 


and it is clear that a special case of (23.48) occurs for a real kernel that is also 
symmetric with respect to its two arguments. 
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Let us begin by considering the homogeneous integral equation 
y=2Ky, 


where the integral operator K has an Hermitian kernel. As discussed in sec- 
tion 23.3, in general this equation will have solutions only for A = 4;, where the 4; 
are the eigenvalues of the integral equation, the corresponding solutions y; being 
the eigenfunctions of the equation. 

By following similar arguments to those presented in chapter 17 for SL theory, 
it may be shown that the eigenvalues /; of an Hermitian kernel are real and 
that the corresponding eigenfunctions y; belonging to different eigenvalues are 
orthogonal and form a complete set. If the eigenfunctions are suitably normalised, 
we have 


b 
Wily) = il yiooy(x) dx = by. (23.49) 


If an eigenvalue is degenerate then the eigenfunctions corresponding to that 
eigenvalue can be made orthogonal by the Gram—Schmidt procedure, in a similar 
way to that discussed in chapter 17 in the context of SL theory. 

Like SL theory, SH theory does not provide a method of obtaining the eigen- 
values and eigenfunctions of any particular homogeneous integral equation with 
an Hermitian kernel; for this we have to turn to the methods discussed in the 
previous sections of this chapter. Rather, SH theory is concerned with the gen- 
eral properties of the solutions to such equations. Where SH theory becomes 
applicable, however, is in the solution of inhomogeneous integral equations with 
Hermitian kernels for which the eigenvalues and eigenfunctions of the corre- 
sponding homogeneous equation are already known. 

Let us consider the inhomogeneous equation 


y=f +iKy, (23.50) 


where K = K? and for which we know the eigenvalues 4; and normalised 
eigenfunctions y; of the corresponding homogeneous problem. The function f 
may or may not be expressible solely in terms of the eigenfunctions y;, and to 
accommodate this situation we write the unknown solution y as y = f + >; aiyi, 
where the a; are expansion coefficients to be determined. 

Substituting this into (23.50), we obtain 


f+ cay; =f+ayo + aKy, (23.51) 


where we have used the fact that y; = A; y;. Forming the inner product of both 
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sides of (23.51) with y,;, we find 


da (y,lyi) a, -(yjlvi) + AYIKE), (23.52) 


Since the eigenfunctions are orthonormal and K is an Hermitian operator, 
we have that both (yjly;) = di and (y|Kf) = (Ky|f) = A;'(yi|f). Thus the 
coefficients aj; are given by 


MGM yilf) _— Ayilf) 


a (23.53) 
1-H; aya 


aj = 


and the solution is 
we lf) 
= ive 5a Ee 23.54 
y ip aiy, ae res (23.54) 


This also shows, incidentally, that a formal representation for the resolvent kernel 
is 


R(x,2;4) = ee tx) aaa (23.55) 


If f can be expressed as a linear superposition of the y;, ie. f = 5°; biy;, then 

b; = (yi|f) and the solution can be written more briefly as 
b; 
y= x Toa (23.56) 

We see from (23.54) that the inhomogeneous equation (23.50) has a unique 
solution provided 2 # /;, i.e. when / is not equal to one of the eigenvalues of 
the corresponding homogeneous equation. However, if 2 does equal one of the 
eigenvalues /; then, in general, the coefficients a; become singular and no (finite) 
solution exists. 

Returning to (23.53), we notice that even if A = 4; a non-singular solution to 
the integral equation is still possible provided that the function f is orthogonal 
to every eigenfunction corresponding to the eigenvalue 4,, i.e. 


b 
(yilf) = { yi(x) f(x) dx = 0. 


The following worked example illustrates the case in which f can be expressed in 
terms of the y;. One in which it cannot is considered in exercise 23.14. 


> Use Schmidt—Hilbert theory to solve the integral equation 


y(x) = sin(x + a) + a sin(x + z)y(z) dz. 
0 


It is clear that the kernel K(x,z) = sin(x +z) is real and symmetric in x and z and is 
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thus Hermitian. In order to solve this inhomogeneous equation using SH theory, however, 
we must first find the eigenvalues and eigenfunctions of the corresponding homogeneous 
equation. 

In fact, we have considered the solution of the corresponding homogeneous equation 
(23.13) already, in subsection 23.4.1, where we found that it has two eigenvalues J; = 2/2 


and A) = —2/z, with eigenfunctions given by (23.16). The normalised eigenfunctions are 
1 1 
yi(x) = ie +cosx) and (x)= wo x — cos x) (23.58) 


and are easily shown to obey the orthonormality condition (23.49). 
Using (23.54), the solution to the inhomogeneous equation (23.57) has the form 


y(x) = aryi(X) + any2(x), (23.59) 


where the coefficients a; and a are given by (23.53) with f(x) = sin(x + «). Therefore, 
using (23.58), 


ae agra ; : 
a= mon ve +cosz)sin(z +a)dz = 5) -(cos a + sin «), 


a = (sin z — cosz)sin(z +a)dz = (cos x — sina). 


24+71 


anf 7 Wid 


Substituting these expressions for a; and a) into (23.59) and simplifying, we find that 
the solution to (23.57) is given by 
1 


y(x) = I aay [sin(x + a) + (14/2) cos(x — @)]. 


23.8 Exercises 


23.1 Solve the integral equation 


ie cos(xv)y(v) dv = exp(—x?/2) 
0 


for the function y = y(x) for x > 0. Note that for x < 0, y(x) can be chosen as 
is most convenient. 
23.2 Solve 


a f(t) exp(—st) dt = Z = Zz 
23.3 Convert 
fis) = expx+ (= viftyydy 
into a differential equation, and hence show that its solution is 
(a+ Bx)expx +) exp(—x), 


where «, f and y are constants that should be determined. 
23.4 Use the fact that its kernel is separable, to solve for y(x) the integral equation 


V(x) = Acos(x +a) + ve sin(x + z)y(z) dz. 
0 


[This equation is an inhomogeneous extension of the homogeneous Fredholm 
equation (23.13), and is similar to equation (23.57). ] 
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23.5 


23.6 


23.7 


Solve for (x) the integral equation 


‘ 1 x n y\n 
oo) =f09+2 f° |(4) +(2)'] oora, 
where f(x) is bounded for 0 < x < 1 and —4 <n< 7 expressing your answer 


in terms of the quantities F,, = i f(y)y" dy. 


(a) Give the explicit solution when 4 = 1. 
(b) For what values of 2 are there no solutions unless F,,, are in a particular 
ratio? What is this ratio? 


Consider the inhomogeneous integral equation 
b 
foe) = e042 f Kooy ifordy. 


for which the kernel K(x,y) is real, symmetric and continuous in a < x < b, 
asy<b. 


(a) If 4 is one of the eigenvalues 1; of the homogeneous equation 


b 
fix) =A ih K(x, y)fily) dy, 


prove that the inhomogeneous equation can only a have non-trivial solution 
if g(x) is orthogonal to the corresponding eigenfunction f;(x). 
(b) Show that the only values of 4 for which 


1 
(0) =2 |) xyi0-+ vgn dy 
0 
has a non-trivial solution are the roots of the equation 
2 + 1204 — 240 = 0. 
(c) Solve 
1 
foe) = wo? + J axy(x + yifrdy. 
0 


The kernel of the integral equation 


b 
w(x) =A / K(x, yoy) dy 
has the form 


K(x,y) = Do In()gn(y), 


n=0 


where the h,(x) form a complete orthonormal set of functions over the interval 
[a, 5]. 


(a) Show that the eigenvalues /; are given by 
|M—A7"l| =0, 


where M is the matrix with elements 
b 
My = | swhy(u)a 


If the corresponding solutions are p(x) = 7°.) ah,(x), find an expression 
for al. 
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23.8 


23.9 


23.10 


23.11 


23.12 


(b) Obtain the eigenvalues and eigenfunctions over the interval [0,27] if 
“1 
K(x, y) = = ), 
(x, y) > 7 COS NX COS ny 


—ax 


By taking its Laplace transform, and that of x"e~“*, obtain the explicit solution 


of 
foxy =e |x [c= merfina 
0 


Verify your answer by substitution. 

For f(t) = exp(—t*/2), use the relationships of the Fourier transforms of f'(t) and 
tf(t) to that of f(t) itself to find a simple differential equation satisfied by fl), 
the Fourier transform of f(t), and hence determine f(@) to within a constant. 
Use this result to solve the integral equation 


io.6) 
/ et25)/2 (7) des 3/8 
—0O 


for h(t). 
Show that the equation 


fixysxP $a | © f(y)exp(—xy) dy 
0 


has a solution of the form Ax% + Bx’, Determine the values of « and B, and show 
that those of A and B are 
1 AT( 
——— and —————— 
1— #2 (4)T(3) 1— 2T( 
where I'(z) is the gamma function. 
At an international ‘peace’ conference a large number of delegates are seated 
around a circular table with each delegation sitting near its allies and diametrically 
opposite the delegation most bitterly opposed to it. The position of a delegate is 
denoted by 0, with 0 < @ < 2z. The fury f(0) felt by the delegate at 0 is the sum 
of his own natural hostility h(@) and the influences on him of each of the other 
delegates; a delegate at position ¢ contributes an amount K(0 — ¢)f(@). Thus 


2n 


f(8) = h(8) + K(0— ¢)f(@)d¢. 
0 
Show that if K(y) takes the form K(wy) = kp +k, cosy then 
f(0) =h(0) +p+qcos0 +rsind 


and evaluate p, q and r. A positive value for k; implies that delegates tend to 
placate their opponents but upset their allies, whilst negative values imply that 
they calm their allies but infuriate their opponents. A walkout will occur if f(@) 
exceeds a certain threshold value for some @. Is this more likely to happen for 
positive or for negative values of k,? 

By considering functions of the form h(x) = Jo (x — y)f(y)dy, show that the 
solution f(x) of the integral equation 


1 
fax+4 i Ix — ylfv) dy 


satisfies the equation f”(x) = f(x). 


821 


INTEGRAL EQUATIONS 


23.13 


23.14 


23.15 


23.16 


By examining the special cases x = 0 and x = 1, show that 


f(x) = [(e + 2)e* — ee]. 


2 
(e+ 3)(e+ 1) 
The operator .@ is defined by 


afio= | Kesy)flvdy, 


where K(x, y) = 1 inside the square |x| < a,|y| < a and K(x,y) = 0 elsewhere. 
Consider the possible eigenvalues of .W and the eigenfunctions that correspond 
to them; show that the only possible eigenvalues are 0 and 2a and determine the 
corresponding eigenfunctions. Hence find the general solution of 


feel Koy flay. 


—oo 


For the integral equation 


b 
y(x) =x + af x°zy(z) dz, 


show that the resolvent kernel is 5x?z?/[5 — (b> — a°)] and hence solve the 
equation. For what range of / is the solution valid? 


Use Fredholm theory to show that, for the kernel 
K(x, z) = (x +z) exp(x — z) 
over the interval [0,1], the resolvent kernel is 


exp(x — z)[(x +z) — M4x + 32 —xXZ— 1) 


R(x,254) = [oe ; 


and hence solve 
1 
ys) =32-+2 [Get 2)explx —2) ve) de, 
0 


expressing your answer in terms of I,,, where I, = fe u" exp(—u) du. 
This exercise shows that following formal theory is not necessarily the best way 
to get practical results! 


(a) Determine the eigenvalues 4+ of the kernel K(x,z) = (xz)!/?(x!/? + 21/2) and 
show that the corresponding eigenfunctions have the forms 


ya(x) = Az(J2x!? + J3x), 


where A}, = 5/(10 + 4,/6). 
(b) Use Schmidt—Hilbert theory to solve 


1 
y(x) =14+ if K(x, z)y(z) dz. 


(c) As will have been apparent, the algebra involved in the formal method used 
in (b) is long and error-prone, and it is in fact much more straightforward 
to use a trial function 1 + wx!/? + Bx. Check your answer by doing so. 
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23.1 


23.3 
23.5 


23.7 


23.9 


23.11 


23.13 


23.15 


23.9 Hints and answers 


Define y(—x) = y(x) and use the cosine Fourier transform inversion theorem; 
Y(x) = (2/7) We exp(—x7/2). 

f(x) — f(x) = exp x; « = 3/4, B = 1/2,» = 1/4. 

(a) d(x) = i) — (1 + 2n)F,x" — (1 — 2n)F_,x~". (b) There are no solutions for 
i. =[1+d— an ed as F,,, =0 or F,/F_» = +[(1 — 2n)/(1 + 2n)]'”. 


a= =f? hy(x)p(x) dx; (b) use (1/,/m) cosnx and (1/,/7) sinnx; M is diago- 
a Pi ee =k/n vs eigenfunctions p(x) = (1/./1) cos kx. 
df /dw = —of, leading to f(@) = Ae~” P/2, Rearrange the integral as a convolution 


and deduce that h(w) = pars h(t) = Ce’, where resubstitution and 
Gaussian normalisation show that C = /2/Bn). 

p = koH (1 — 2nko), q = kiHe/(1 — why) and r = ky H,/(1 — ky), 

where H = ie h(z) dz, H, = te h(z)cosz dz, and H, = ie h(z) sin z dz. Positive 
values of k,(~ 27!) are most likely to cause a conference breakdown. 

For eigenvalue 0 : f(x) = 0 for |x| <a or f(x) is such that ye fiy)dy =0. 

For eigenvalue 2a : f(x) = uS(x,a) with p a constant and S(x,a) = [H(a+ x) — 
A(x —a)], ere H(z) is the Heaviside step function. 

Take f( a (x) + cGS(x, a), where G = g(z) dz. Show that c = 1/(1 — 2ad). 
y(x) = x? — ae + I) exp x. 
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Throughout this book references have been made to results derived from the the- 
ory of complex variables. This theory thus becomes an integral part of the mathe- 
matics appropriate to physical applications. Indeed, so numerous and widespread 
are these applications that the whole of the next chapter is devoted to a systematic 
presentation of some of the more important ones. This current chapter develops 
the general theory on which these applications are based. The difficulty with it, 
from the point of view of a book such as the present one, is that the underlying 
basis has a distinctly pure mathematics flavour. 

Thus, to adopt a comprehensive rigorous approach would involve a large 
amount of groundwork in analysis, for example formulating precise definitions 
of continuity and differentiability, developing the theory of sets and making a 
detailed study of boundedness. Instead, we will be selective and pursue only those 
parts of the formal theory that are needed to establish the results used in the next 
chapter and elsewhere in this book. 

In this spirit, the proofs that have been adopted for some of the standard 
results of complex variable theory have been chosen with an eye to simplicity 
rather than sophistication. This means that in some cases the imposed conditions 
are more stringent than would be strictly necessary if more sophisticated proofs 
were used; where this happens the less restrictive results are usually stated as 
well. The reader who is interested in a fuller treatment should consult one of the 
many excellent textbooks on this fascinating subject 

One further concession to ‘hand-waving’ has been made in the interests of 
keeping the treatment to a moderate length. In several places phrases such as ‘can 
be made as small as we like’ are used, rather than a careful treatment in terms 
of ‘given e > 0, there exists a 6 > 0 such that’. In the authors’ experience, some 


8 For example, K. Knopp, Theory of Functions, Part I (New York: Dover, 1945); E. G. Phillips, 
Functions of a Complex Variable with Applications 7th edn (Edinburgh: Oliver and Boyd, 1951); E. 
C. Titchmarsh, The Theory of Functions (Oxford: Oxford University Press, 1952). 
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students are more at ease with the former type of statement, despite its lack of 
precision, whilst others, those who would contemplate only the latter, are usually 
well able to supply it for themselves. 


24.1 Functions of a complex variable 


The quantity f(z) is said to be a function of the complex variable z if to every value 
of z in a certain domain R (a region of the Argand diagram) there corresponds 
one or more values of f(z). Stated like this f(z) could be any function consisting 
of a real and an imaginary part, each of which is, in general, itself a function of x 
and y. If we denote the real and imaginary parts of f(z) by u and v, respectively, 
then 


f(z) = u(x, y) + iv(x, y). 


In this chapter, however, we will be primarily concerned with functions that are 
single-valued, so that to each value of z there corresponds just one value of f(z), 
and are differentiable in a particular sense, which we now discuss. 

A function f(z) that is single-valued in some domain R is differentiable at the 
point z in R if the derivative 


(24.1) 


f'(e) = fim, ie + Az) — Lay 


Az 


exists and is unique, in that its value does not depend upon the direction in the 
Argand diagram from which Az tends to zero. 


> Show that the function f(z) = x* — y* + i2xy is differentiable for all values of z. 


Considering the definition (24.1), and taking Az = Ax + iAy, we have 
f(z + Az) — f(z) 


Az 

_ + AXP = (y + Ay? + 2i(x + Ax(y + Ay) — x? + y? = 2ixy 
Ax + iAy 

_ 2xAx + (Ax)? — 2yAy — (Ay)? + 2i(xAy + yAx + AxAy) 

a Ax + iAy 

_ ; (Ax)? — (Ay)? + 2iAxAy 

a REID Pt — erage = t : 


Now, in whatever way Ax and Ay are allowed to tend to zero (e.g. taking Ay = 0 and 
letting Ax — 0 or vice versa), the last term on the RHS will tend to zero and the unique 
limit 2x + i2y will be obtained. Since z was arbitrary, f(z) with u = x? — y* and v = 2xy 
is differentiable at all points in the (finite) complex plane. < 


We note that the above working can be considerably reduced by recognising 
that, since z = x + iy, we can write f(z) as 


f(z) =x? —y? + 2ixy =(x + iy = 27. 
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We then find that 


= lim 


Az—>0 Az 


- ( Jim, Az) +22 =2z, 


z+Azy?— =) [ast + — 


from which we see immediately that the limit both exists and is independent of 
the way in which Az — 0. Thus we have verified that f(z) = z? is differentiable 
for all (finite) z. We also note that the derivative is analogous to that found for 
real variables. 

Although the definition of a differentiable function clearly includes a wide 
class of functions, the concept of differentiability is restrictive and, indeed, some 
functions are not differentiable at any point in the complex plane. 


> Show that the function f(z) = 2y+ix is not differentiable anywhere in the complex plane. 


In this case f(z) cannot be written simply in terms of z, and so we must consider the 
limit (24.1) in terms of x and y explicitly. Following the same procedure as in the previous 
example we find 


f(z +Az)—f(z) — 2y +2Ay + ix +iAx — 2y — ix 
Az = Ax + iAy 
_ 2Ay + iAx 
~ Ax + iAy* 


In this case the limit will clearly depend on the direction from which Az —> 0. Suppose 
Az — 0 along a line through z of slope m, so that Ay = mAx, then 


iy f(z + Az) — f(z) ef 2Ay es - cue 
Az—0 Az Ax, Ay0 | Ax + iAy 1+im 


This limit is dependent on m and hence on the direction from which Az — 0. Since this 
conclusion is independent of the value of z, and hence true for all z, f(z) = 2y + ix is 
nowhere differentiable. < 


A function that is single-valued and differentiable at all points of a domain R 
is said to be analytic (or regular) in R. A function may be analytic in a domain 
except at a finite number of points (or an infinite number if the domain is 
infinite); in this case it is said to be analytic except at these points, which are 
called the singularities of f(z). In our treatment we will not consider cases in 
which an infinite number of singularities occur in a finite domain. 
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> Show that the function f(z) = 1/(1 —2z) is analytic everywhere except at z = 1. 


Since f(z) is given explicitly as a function of z, evaluation of the limit (24.1) is somewhat 
easier. We find 


F 1 1 
ADO la —z—Az\l— 5 (—2zy’ 


independently of the way in which Az — 0, provided z # 1. Hence f(z) is analytic 
everywhere except at the singularity z = 1. < 


24.2 The Cauchy—Riemann relations 


From examining the previous examples, it is apparent that for a function f(z) 
to be differentiable and hence analytic there must be some particular connection 
between its real and imaginary parts u and v. 
By considering a general function we next establish what this connection must 
be. If the limit 
pa VetIEfo 


Az—0 Az Ce 


is to exist and be unique, in the way required for differentiability, then any two 
specific ways of letting Az — 0 must produce the same limit. In particular, moving 
parallel to the real axis and moving parallel to the imaginary axis must do so. 
This is certainly a necessary condition, although it may not be sufficient. 

If we let f(z) = u(x, y) + iv(x, y) and Az = Ax + iAy then we have 


f(z + Az) = u(x + Ax, y + Ay) + iv(x + Ax, y + Ay), 
and the limit (24.2) is given by 
pe, 4h “ + Ax, y + Ay) + iv(x + Ax, y + Ay) — u(x, y) — “ey 
Ax, Ay0 Ax + iAy 
If we first suppose that Az is purely real, so that Ay = 0, we obtain 
i“ + Ax, y) — u(x, y) i ix + Ax, y)— en) _ ou Fs 00, 
Ax Ax Ox (24.3) 


L= lim = 
Ax-0 


provided each limit exists at the point z. Similarly, if Az is taken as purely 
imaginary, so that Ax = 0, we find 


: u(x,y + Ay)—u(x,y) v(x, y + Ay) — v(x, y) lou dv 
= fim, A Lj x = ae 
= ny ne y a44) 


io 


827 


COMPLEX VARIABLES 


For f to be differentiable at the point z, expressions (24.3) and (24.4) must 
be identical. It follows from equating real and imaginary parts that necessary 
conditions for this are 
3 
OP cng (24.5) 

Ox oy Ox oy 
These two equations are known as the Cauchy—Riemann relations. 

We can now see why for the earlier examples (i) f(z) = x? — y? + i2xy might 
be differentiable and (ii) f(z) = 2y + ix could not be. 


(i) u=x?—y?, v = 2xy: 


OF ie Ae and ME 2 oy a 
Ox oy Ox oy 
(il) u=2y,v =x: 
ou Ov ov ou 
— =0=— but —=14-2=-—. 
Ox oy Ox r oy 


It is apparent that for f(z) to be analytic something more than the existence 
of the partial derivatives of u and v with respect to x and y is required; this 
something is that they satisfy the Cauchy—Riemann relations. 

We may enquire also as to the sufficient conditions for f(z) to be analytic in 
R. It can be shown’ that a sufficient condition is that the four partial derivatives 
exist, are continuous and satisfy the Cauchy—Riemann relations. It is the addi- 
tional requirement of continuity that makes the difference between the necessary 
conditions and the sufficient conditions. 


>In which domain(s) of the complex plane is f(z) = |x| — ily| an analytic function? 


Writing f = u + iv it is clear that both du/dy and dv/éx are zero in all four quadrants 
and hence that the second Cauchy—Riemann relation in (24.5) is satisfied everywhere. 

Turning to the first Cauchy—Riemann relation, in the first quadrant (x > 0, y > 0) we 
have f(z) = x —iy so that 


ou ov 


= =1, —=-l, 
Ox oy 
which clearly violates the first relation in (24.5). Thus f(z) is not analytic in the first 
quadrant. 
Following a similiar argument for the other quadrants, we find 


es =-1 or +1. forx<0Oand x >0, respectively, 
Ox 

ov . 

ay =-1 or +1  fory>Oand y <0, respectively. 
6 


Therefore du/0x and 0v/dy are equal, and hence f(z) is analytic only in the second and 
fourth quadrants. < 


S See, for example, any of the references given on page 824. 
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Since x and y are related to z and its complex conjugate z* by 
1 1 
x= <(z+2z") and y= —(z—-2’'), (24.6) 
2 2i 


we may formally regard any function f =u-+ iv as a function of z and z*, rather 
than x and y. If we do this and examine df /0z" we obtain 
Of _ of ox | of dy 


0z* O0x0dz*  6ydz* 


Le 1 ss au 1 
ax ax) \2 dy | dy i 
1 fou ov i (dv | ou 
=-(—-— =~{—+-—). 24.7 
(+ ~)+5(S+3) ee 
Now, if f is analytic then the Cauchy—Riemann relations (24.5) must be satisfied, 
and these immediately give that df /0z" is identically zero. Thus we conclude that 
if f is analytic then f cannot be a function of z* and any expression representing 


an analytic function of z can contain x and y only in the combination x + iy, not 
in the combination x — iy. 


We conclude this section by discussing some properties of analytic functions 
that are of great practical importance in theoretical physics. These can be obtained 
simply from the requirement that the Cauchy—Riemann relations must be satisfied 
by the real and imaginary parts of an analytic function. 

The most important of these results can be obtained by differentiating the 
first Cauchy—Riemann relation with respect to one independent variable, and the 
second with respect to the other independent variable, to obtain the two chains 
of equalities 


0 (du\ 0 (a\ 8 (av\__ a (au 

dx Lax) dx \dy) dy \éx) ay \ ay)’ 

0 (a\ 0 (au) __ 0 (au) __ a (a 
ax Lax) dx \ay} ay x} ay \ ay)” 


Thus both u and v are separately solutions of Laplace’s equation in two dimen- 
sions, Le. 

u  e?u Pv Orv 

—+—=0 and —+=— =0. 24.8 

6x? dy? 6x2 dy? axe] 
We will make significant use of this result in the next chapter. 

A further useful result concerns the two families of curves u(x, y) = constant 
and v(x, y) = constant, where u and v are the real and imaginary parts of any 
analytic function f = u+iv. As discussed in chapter 10, the vector normal to the 
curve u(x, y) = constant is given by 


ou, ou, 
Vu= sits (24.9) 
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where i and j are the unit vectors along the x- and y-axes, respectively. A similar 
expression exists for Vv, the normal to the curve v(x, y) = constant. Taking the 
scalar product of these two normal vectors, we obtain 


Ou dv. du dv 

Ox0x oy doy 
dudu  Oudu 
Oxdy 6ydx ~ 


where in the last line we have used the Cauchy—Riemann relations to rewrite 
the partial derivatives of v as partial derivatives of u. Since the scalar product 
of the normal vectors is zero, they must be orthogonal, and the curves u(x, y) = 
constant and v(x, y) = constant must therefore intersect at right angles. 


> Use the Cauchy—Riemann relations to show that, for any analytic function f = u-+iv, the 
relation |Vu| = |Vv| must hold. 


From (24.9) we have 


2 au\? du\? 
\Vul Vu- Vu ax + a 


Using the Cauchy—Riemann relations to write the partial derivatives of u in terms of those 


of v, we obtain 
av\2 av\2 
\Vul2 (=) +( =) Vor, 
oy Ox 


from which the result |Vu| = |Vv| follows immediately. < 


24.3 Power series in a complex variable 


The theory of power series in a real variable was considered in chapter 4, which 
also contained a brief discussion of the natural extension of this theory to a series 
such as 


ioe) 
IO az (24.10) 
n=0 
where z is a complex variable and the a, are, in general, complex. We now 
consider complex power series in more detail. 

Expression (24.10) is a power series about the origin and may be used for 
general discussion, since a power series about any other point zo can be obtained 
by a change of variable from z to z — zo. If z were written in its modulus and 
argument form, z = rexpi0, expression (24.10) would become 


a 


f(z) = Yo apr” exp(ind). (24.11) 


n=0 
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This series is absolutely convergent if 


ioe) 

SO lanlr”, (24.12) 

n=0 
which is a series of positive real terms, is convergent. Thus tests for the absolute 
convergence of real series can be used in the present context, and of these the 
most appropriate form is based on the Cauchy root test. With the radius of 
convergence R defined by 


no 


aes 
R 7 lim la,|'/", (24.13) 


the series (24.10) is absolutely convergent if |z| < R and divergent if |z| > R. 
If |z| = R then no particular conclusion may be drawn, and this case must be 
considered separately, as discussed in subsection 4.5.1. 

A circle of radius R centred on the origin is called the circle of convergence 
of the series 5“ a,z". The cases R = 0 and R = ~ correspond, respectively, to 
convergence at the origin only and convergence everywhere. For R finite the 
convergence occurs in a restricted part of the z-plane (the Argand diagram). For 
a power series about a general point zo, the circle of convergence is, of course, 
centred on that point. 


> Find the parts of the z-plane for which the following series are convergent: 


co ioe) gn 
(ii) Sontz", iii) SO =. 
n=0 n=1 Me 


(i) Since (n!)!/" behaves like n as n — co we find lim(1/n!)'/" = 0. Hence R = co and the 
series is convergent for all z. (ii) Correspondingly, lim(n!)!/" = oo. Thus R = 0 and the 
series converges only at z = 0. (iii) As n > 00, (n)'/" has a lower limit of 1 and hence 
lim(1/n)!/" = 1/1 = 1. Thus the series is absolutely convergent if the condition |z| < 1 is 
satisfied. < 


Case (iii) in the above example provides a good illustration of the fact that 
on its circle of convergence a power series may or may not converge. For this 
particular series, the circle of convergence is |z| = 1, so let us consider the 
convergence of the series at two different points on this circle. Taking z = 1, the 
series becomes 

on ei ee 
DS itaita ees 


n=1 


which is easily shown to diverge (by, for example, grouping terms, as discussed in 
subsection 4.3.2). Taking z = —1, however, the series is given by 


ye ite! 
n "2 3°4 : 


n=1 
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which is an alternating series whose terms decrease in magnitude and which 
therefore converges. 

The ratio test discussed in subsection 4.3.2 may also be employed to investi- 
gate the absolute convergence of a complex power series. A series is absolutely 
convergent if 


finn @mtellel! nel 


nn |ay||2|" noe dnl 


2A (24.14) 


and hence the radius of convergence R of the series is given by 


1 . |a 
= lim | n+1| 


R ne || ; 
For instance, in case (i) of the previous example, we have 


n! 


R Seen! a 


Thus the series is absolutely convergent for all (finite) z, confirming the previous 
result. 

Before turning to particular power series, we conclude this section by stating 
the important result’ that the power series S7f dnz" has a sum that is an analytic 
function of z inside its circle of convergence. 

As a corollary to the above theorem, it may further be shown that if f(z) = 
2 a,z" then, inside the circle of convergence of the series, 


me 
f'(2) = »y; nayz" 
n=0 


Repeated application of this result demonstrates that any power series can be 
differentiated any number of times inside its circle of convergence. 


24.4 Some elementary functions 


In the example at the end of the previous section it was shown that the function 
expz defined by 


ee) 
Zz” 


expz=)_ a (24.15) 


n=0 


is convergent for all z of finite modulus and is thus, by the discussion of 
the previous section, an analytic function over the whole z-plane.1 Like its 


S For a proof see, for example, K. F. Riley, Mathematical Methods for the Physical Sciences (Cam- 
bridge: Cambridge University Press, 1974), p. 446. 


4 Functions that are analytic in the whole z-plane are usually called integral or entire functions. 
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real-variable counterpart it is called the exponential function; also like its real 
counterpart it is equal to its own derivative. 

The multiplication of two exponential functions results in a further exponential 
function, in accordance with the corresponding result for real variables. 


> Show that exp z; exp Z2 = exp(z; + 22). 


From the series expansion (24.15) of expz, and a similar expansion for exp Zo, it is clear 
that the coefficient of zjz5 in the corresponding series expansion of exp Zz) exp Z2 is simply 
1/(ris!). 

But, from (24.15) we also have 


oe) 


(z, + 22) 
exp(Z; + 22) = ~—— : 2) 


al 
n=0 


In order to find the coefficient of z}z5 in this expansion, we clearly have to consider the 
term in which n =r +s, namely 
tay 1 
(r+s)! (r+s)! 
The coefficient of z/z} in this is given by 
50 1 (r+s)! 1 1 
“(r+ 5)! sir! (r+s)! rls! 
Thus, since the corresponding coefficients on the two sides are equal, and all the series 


involved are absolutely convergent for all z, we can conclude that exp z, exp Z. = exp(z; + 
22). < 


(“Cozy Heese "4S Cozi zs pets al SG igaz5**) . 


As an extension of (24.15) we may also define the complex exponent of a real 
number a > 0 by the equation 


a = exp(z Ina), (24.16) 


where Ina is the natural logarithm of a. The particular case a = e and the fact 
that Ine = 1 enable us to write expz interchangeably with e’. If z is real then the 
definition agrees with the familiar one. 

The result for z = iy, 


expiy = cosy +isiny, (24.17) 
has been met already in equation (3.23). Its immediate extension is 
expz = (exp x)(cos y + isin y). (24.18) 


As z varies over the complex plane, the modulus of expz takes all real positive 
values, except that of 0. However, two values of z that differ by 2zki, for any 
integer k, produce the same value of expz, as given by (24.18), and so expz is 
periodic with period 2zi. If we denote expz by t, then the strip —7 <y <a in 
the z-plane corresponds to the whole of the t-plane, except for the point t = 0. 

The sine, cosine, sinh and cosh functions of a complex variable are defined from 
the exponential function exactly as are those for real variables. The functions 
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derived from them (e.g. tan and tanh), the identities they satisfy and their 
derivative properties are also just as for real variables. In view of this we will not 
give them further attention here. 

The inverse function of expz is given by w, the solution of 


expw =z. (24.19) 


This inverse function was discussed in chapter 3, but we mention it again here 
for completeness. By virtue of the discussion following (24.18), w is not uniquely 
defined and is indeterminate to the extent of any integer multiple of 27i. If we 
express Z as 


z =rexpi0, 


where r is the (real) modulus of z and @ is its argument (—z < 0 < 2), then 
multiplying z by exp(2ikz), where k is an integer, will result in the same complex 
number z. Thus we may write 


z =rexp[i(@ + 2kz)], 


where k is an integer. If we denote w in (24.19) by 


w =Lnz=Inr+i(0 + 2kn), (24.20) 


where Inr is the natural logarithm (to base e) of the real positive quantity r, then 
Lnz is an infinitely multivalued function of z. Its principal value, denoted by Inz, 
is obtained by taking k = 0 so that its argument lies in the range —z to x. Thus 


Inz=Inr+i0, with-z<0<xn. (24.21) 


Now that the logarithm of a complex variable has been defined, definition (24.16) 
of a general power can be extended to cases other than those in which a is real 
and positive. If t (4 0) and z are both complex, then the zth power of t is defined 
by 


t? = exp(z Ln 2). (24.22) 


Since Lnt is multivalued, so is this definition. Its principal value is obtained by 
giving Lnt its principal value, Int. 

If t (# 0) is complex but z is real and equal to 1/n, then (24.22) provides a 
definition of the nth root of t. Because of the multivaluedness of Ln t, there will 
be more than one nth root of any given t. 


>» Show that there are exactly n distinct nth roots of t. 


From (24.22) the nth roots of t are given by 


1 
1/n __ 
r= -Lnt). 
exp (< n ) 
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On the RHS let us write t as follows: 
t=rexp[i(@ + 2kz)], 


where k is an integer. We then obtain 


1 6 + 2k 
fn = exp F Inr+ jena) zz — 
n n 
sie [ie + =) 
n 
where k = 0,1,...,n —1; for other values of k we simply recover the roots already found. 


Thus t has n distinct nth roots. < 


24.5 Multivalued functions and branch cuts 


In the definition of an analytic function, one of the conditions imposed was 
that the function is single-valued. However, as shown in the previous section, the 
logarithmic function, a complex power and a complex root are all multivalued. 
Nevertheless, it happens that the properties of analytic functions can still be 
applied to these and other multivalued functions of a complex variable provided 
that suitable care is taken. This care amounts to identifying the branch points of 
the multivalued function f(z) in question. If z is varied in such a way that its 
path in the Argand diagram forms a closed curve that encloses a branch point, 
then, in general, f(z) will not return to its original value. 

For definiteness let us consider the multivalued function f(z) = z!/? and express 
z as z =rexpi0. From figure 24.1(a), it is clear that, as the point z traverses any 
closed contour C that does not enclose the origin, @ will return to its original 
value after one complete circuit. However, for any closed contour C’ that does 
enclose the origin, after one circuit 0 > 0 + 27 (see figure 24.1(b)). Thus, for the 
function f(z) = z!/?, after one circuit 


r/? exp(i0/2) > r!/? exp[i(O + 22)/2] = —r'/? exp(i0/2). 


In other words, the value of f(z) changes around any closed loop enclosing the 
origin; in this case f(z) — —f(z). Thus z = 0 is a branch point of the function 
f(z) =2'/?, 

We note in this case that if any closed contour enclosing the origin is traversed 
twice then f(z) = z!/? returns to its original value. The number of loops around 
a branch point required for any given function f(z) to return to its original value 
depends on the function in question, and for some functions (e.g. Ln z, which also 
has a branch point at the origin) the original value is never recovered. 

In order that f(z) may be treated as single-valued, we may define a branch cut 
in the Argand diagram. A branch cut is a line (or curve) in the complex plane 
and may be regarded as an artificial barrier that we must not cross. Branch cuts 
are positioned in such a way that we are prevented from making a complete 
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y ¥ yy 


(a) (b) (c) 


Figure 24.1 (a) A closed contour not enclosing the origin; (b) a closed 
contour enclosing the origin; (c) a possible branch cut for f(z) = z!/?. 


circuit around any one branch point, and so the function in question remains 
single-valued. 

For the function f(z) = z'/?, we may take as a branch cut any curve starting 
at the origin z = 0 and extending out to |z| = «© in any direction, since all 
such curves would equally well prevent us from making a closed loop around 
the branch point at the origin. It is usual, however, to take the cut along either 
the real or the imaginary axis. For example, in figure 24.1(c), we take the cut as 
the positive real axis. By agreeing not to cross this cut, we restrict 0 to lie in the 
range 0 < 0 < 2z, and so keep f(z) single-valued. 

These ideas are easily extended to functions with more than one branch point. 


> Find the branch points of f(z) = z* +1, and hence sketch suitable arrangements of 
branch cuts. 


We begin by writing f(z) as 


f2@=V24+1= Je—ie +3). 


As shown above, the function g(z) = z'/? has a branch point at z = 0. Thus we might 
expect f(z) to have branch points at values of z that make the expression under the square 
root equal to zero, i.e. at z =i and z = —i. 

As shown in figure 24.2(a), we use the notation 


z—i=r,expi0, and z+i=rzexpi02. 
We can therefore write f(z) as 


f(z) = riz exp(i0; /2) exp(i02/2) = /rirz exp [i(0; + 02)/2] . 


Let us now consider how f(z) changes as we make one complete circuit around various 
closed loops C in the Argand diagram. If C encloses 


(i) neither branch point, then 0; > 01, 02 — 02 and so f(z) > f(z); 
(ii) z =i but not z = —i, then 0, — 0, + 27, 02 — @2 and so f(z) > —f(z); 
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(a) (b) (c) 


Figure 24.2 (a) Coordinates used in the analysis of the branch points of 
f(z) = (22 + 1)!”; (b) one possible arrangement of branch cuts; (c) another 
possible branch cut, which is finite. 


(iii) z = —i but not z =i, then 0, > 6,, 0. > 0, + 2 and so f(z) > —f(z); 

(iv) both branch points, then 0; > 0; + 22, 02 > 02 + 2a and so f(z) > f(z). 

Thus, as expected, f(z) changes value around loops containing either z = i or z = —i (but 
not both). We must therefore choose branch cuts that prevent us from making a complete 
loop around either branch point; one suitable choice is shown in figure 24.2(b). 

For this f(z), however, we have noted that after traversing a loop containing both branch 
points the function returns to its original value. Thus we may choose an alternative, finite, 
branch cut that allows this possibility but still prevents us from making a complete loop 
around just one of the points. A suitable cut is shown in figure 24.2(c). < 


24.6 Singularities and zeros of complex functions 


A singular point of a complex function f(z) is any point in the Argand diagram 
at which f(z) fails to be analytic. We have already met one sort of singularity, 
the branch point, and in this section we will consider other types of singularity 
as well as discuss the zeros of complex functions. 

If f(z) has a singular point at z = zo but is analytic at all points in some 
neighbourhood containing zp but no other singularities, then z = Zo is called an 
isolated singularity. (Clearly, branch points are not isolated singularities.) 

The most important type of isolated singularity is the pole. If f(z) has the form 


f(Z)= sl) — ; (24.23) 
(z — 20)" 
where n is a positive integer, g(z) is analytic at all points in some neighbourhood 
containing z = Zo and g(zo) # 0, then f(z) has a pole of order n at z = zo. An 
alternative (though equivalent) definition is that 


lim [(z — zo)" f(z)] =a, (24.24) 
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where a is a finite, non-zero complex number. We note that if the above limit is 
equal to zero, then z = Zo is a pole of order less than n, or f(z) is analytic there; 
if the limit is infinite then the pole is of an order greater than n. It may also be 
shown that if f(z) has a pole at z = zo, then |f(z)| ~ «0 as z — zo from any 
direction in the Argand diagram5 If no finite value of n can be found such that 
(24.24) is satisfied, then z = zo is called an essential singularity. 


> Find the singularities of the functions 


(i) If we write f(z) as 


1 1 2z 
We) —T 5 Tee Gada 
we see immediately from either (24.23) or (24.24) that f(z) has poles of order 1 (or simple 
poles) atz=1andz=-1. 
(ii) In this case we write 
sinhz _ expz —exp(—z) 


coshz  expz+exp(—z)’ 


f(z) =tanhz = 


Thus f(z) has a singularity when exp z = —exp(—z) or, equivalently, when 
expz = exp[i(2n + 1)z] exp(—z), 


where n is any integer. Equating the arguments of the exponentials we find z = (n+ 3)ni, 
for integer n. 
Furthermore, using I’H6pital’s rule (see chapter 4) we have 


: (Se sme 
lim Ss 


29(n+4)zi coshz 


: { [z-—(n+ 5)ni] soe ae 
= lim : 1. 
sinh z 


29(n+5)xi 


Therefore, from (24.24), each singularity is a simple pole. < 


Another type of singularity exists at points for which the value of f(z) takes 
an indeterminate form such as 0/0 but lim,_,,, f(z) exists and is independent 
of the direction from which zo is approached. Such points are called removable 
singularities. 


> Show that f(z) = (sinz)/z has a removable singularity at z = 0. 


It is clear that f(z) takes the indeterminate form 0/0 at z = 0. However, by expanding 
sinz as a power series in z, we find 


1 ZZ zz 
f@) (2 3 T -) eat St 


S Although perhaps intuitively obvious, this result really requires formal demonstration by analysis. 
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Thus lim,_,o f(z) = 1 independently of the way in which z —> 0, and so f(z) has a removable 
singularity at z =0. < 

An expression common in mathematics, but which we have so far avoided 
using explicitly in this chapter, is ‘z tends to infinity’. For a real variable such 
as |z| or R, ‘tending to infinity’ has a reasonably well defined meaning. For a 
complex variable needing a two-dimensional plane to represent it, the meaning is 
not intrinsically well defined. However, it is convenient to have a unique meaning 
and this is provided by the following definition: the behaviour of f(z) at infinity 
is given by that of f(1/€) at € =0, where € = 1/z. 


> Find the behaviour at infinity of (i) f(z) = a+ bz~*, (ii) f(z) = z(1 + 27) and (iii) 
f(z) =expz. 


(i) f(z) = a+ bz: on putting z = 1/é, f(1/é) = a+ bé?, which is analytic at € = 0; 

thus f is analytic at z = ©. 

(ii) f(z) = z(1 +27): f(1/é) = 1/€ + 1/€3; thus f has a pole of order 3 at z =o. 

(iii) f(z) = expz : f(1/E) = p(n!) 1E; thus f has an essential singularity at z = <0. 4 

We conclude this section by briefly mentioning the zeros of a complex function. 
As the name suggests, if f(zo) = 0 then z = zo is called a zero of the function 
f(z). Zeros are classified in a similar way to poles, in that if 


f(z) = (z — 20)"g(z), 


where n is a positive integer and g(zo) # 0, then z = Zo is called a zero of order 
n of f(z). If n = 1 then z = Zp is called a simple zero. It may further be shown 
that if z = zo is a zero of order n of f(z) then it is also a pole of order n of the 
function 1/f(z). 

We will return in section 24.11 to the classification of zeros and poles in terms 
of their series expansions. 


24.7 Conformal transformations 


We now turn our attention to the subject of transformations, by which we mean 
a change of coordinates from the complex variable z = x + iy to another, say 
w =r -+is, by means of a prescribed formula: 


w = g(Z) =r(x, y) + is(x, y). 


Under such a transformation, or mapping, the Argand diagram for the z-variable 
is transformed into one for the w-variable, although the complete z-plane might 
be mapped onto only a part of the w-plane, or onto the whole of the w-plane, or 
onto some or all of the w-plane covered more than once. 

We shall consider only those mappings for which w and z are related by a 
function w = g(z) and its inverse z = h(w) with both functions analytic, except 
possibly at a few isolated points; such mappings are called conformal. Their 
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A ae \ a 


Figure 24.3. Two curves C; and C, in the z-plane, which are mapped onto 
C; and C; in the w-plane. 


important properties are that, except at points at which g’(z), and hence h'(z), is 
zero or infinite: 


(i) continuous lines in the z-plane transform into continuous lines in the 
w-plane; 

(ii) the angle between two intersecting curves in the z-plane equals the angle 
between the corresponding curves in the w-plane; 

(iii) the magnification, as between the z-plane and the w-plane, of a small line 
element in the neighbourhood of any particular point is independent of 
the direction of the element; 

(iv) any analytic function of z transforms to an analytic function of w and 
vice versa. 


Result (i) is immediate, and results (ii) and (ili) can be justified by the following 
argument. Let two curves C; and C) pass through the point zo in the z-plane 
and let z; and zz be two points on their respective tangents at zo, each a distance 
p from zo. The same prescription with w replacing z describes the transformed 
situation; however, the transformed tangents may not be straight lines and the 
distances of w; and w2 from wo have not yet been shown to be equal. This 
situation is illustrated in figure 24.3. 
In the z-plane z; and z2 are given by 


Z1 — Zo = pexpi0; and Zo — Zo = pexpi0o. 
The corresponding descriptions in the w-plane are 
Wi — Wo = pi expidr and W2 — Wo = p2 expido. 


The angles 0; and ¢; are clear from figure 24.3. The transformed angles ¢; are 
those made with the r-axis by the tangents to the transformed curves at their 
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point of intersection. Since any finite-length tangent may be curved, w; is more 
strictly given by w; — wo = piexpi(d; + 6¢;), where 6d; > 0 as p; > O, Le. as 
po. 

Now since w = g(z), where g is analytic, we have 


‘ W, — Wo W2 — Wo d 
lim = lim a 
2120 \ zy — zo 220 \ zy — 9 dz 


which may be written as 


tim { 2 expticds + 56 ~0))b = tim { explitds + 362 ~0a)1} = e'. 
p—0 p p-0 p (24.25) 
Comparing magnitudes and phases (i.e. arguments) in the equalities (24.25) gives 
the stated results (11) and (iii) and adds quantitative information to them, namely 
that for small line elements 


Pr pw Pw |9'(zo)l, (24.26) 
pp 
$1 — 01 © $2 — 02 & arg g’ (zo). (24.27) 


For strict comparison with result (ii), (24.27) must be written as 0; —02 = ¢1—¢2, 
with an ordinary equality sign, since the angles are only defined in the limit 
p — 0 when (24.27) becomes a true identity. We also see from (24.26) that 
the linear magnification factor is |g’(zo)|; similarly, small areas are magnified by 
Ig’(zo) |’. 

Since in the neighbourhoods of corresponding points in a transformation angles 
are preserved and magnifications are independent of direction, it follows that small 
plane figures are transformed into figures of the same shape, but, in general, ones 
that are magnified and rotated (though not distorted). However, we also note 
that at points where g’(z) = 0, the angle arg g’(z) through which line elements are 
rotated is undefined; these are called critical points of the transformation. 

The final result (iv) is perhaps the most important property of conformal 
transformations. If f(z) is an analytic function of z and z = h(w) is also analytic, 
then F(w) = f(h(w)) is analytic in w. Its importance lies in the further conclusions 
it allows us to draw from the fact that, since f is analytic, the real and imaginary 
parts of f = @ + iy are necessarily solutions of 


— + and = 

0x2 Oy? 0x2 dy? 
Since the transformation property ensures that F = ® + i is also analytic, we 
can conclude that its real and imaginary parts must themselves satisfy Laplace’s 
equation in the w-plane: 

fo oo Oy Ow 


Bre as =0 and "aad + Fst = 0, (24.29) 


BY sO 24 a ea ae (24.28) 
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Figure 24.4 Transforming the upper half of the z-plane into the interior of 
the unit circle in the w-plane, in such a way that z =i is mapped onto w = 0 


and the points x = +oo are mapped onto w = 1. 


Further, suppose that (say) Re f(z) = @ is constant over a boundary C in the 
z-plane; then Re F(w) = ® is constant over C in the z-plane. But this is the same 
as saying that Re F(w) is constant over the boundary C’ in the w-plane, C’ being 
the curve into which C is transformed by the conformal transformation w = g(z). 
This result is exploited extensively in the next chapter to solve Laplace’s equation 
for a variety of two-dimensional geometries. 

Examples of useful conformal transformations are numerous. For instance, 
w =z+b, w =(expi¢)z and w = az correspond, respectively, to a translation by 
b, a rotation through an angle ¢ and a stretching (or contraction) in the radial 
direction (for a real). These three examples can be combined into the general 
linear transformation w = az +b, where, in general, a and b are complex. Another 
example is the inversion mapping w = 1/z, which maps the interior of the unit 
circle to the exterior and vice versa. Other, more complicated, examples also exist. 


> Show that if the point Zo lies in the upper half of the z-plane then the transformation 


Z—Z 
w = (expip) — 
Z— 2 
maps the upper half of the z-plane into the interior of the unit circle in the w-plane. Hence 
find a similar transformation that maps the point z =i onto w = 0 and the points x = +00 
onto w = 1. 


Taking the modulus of w, we have 


2:26 Z—Zo 


|w| = |(expid) = 


P r 
Z—2% Z—2 


However, since the complex conjugate z is the reflection of zo in the real axis, if z and zo 
both lie in the upper half of the z-plane then |z — zo| < |z — z9|; thus |w| < 1, as required. 
We also note that (i) the equality holds only when z lies on the real axis, and so this axis 
is mapped onto the boundary of the unit circle in the w-plane; (ii) the point zo is mapped 
onto w = 0, the origin of the w-plane. 

By fixing the images of two points in the z-plane, the constants zo and @ can also be 
fixed. Since we require the point z = i to be mapped onto w = 0, we have immediately 
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Figure 24.5 Transforming the upper half of the z-plane into the interior 


of a polygon in the w-plane, in such a way that the points x1, x2,...,xX, are 
mapped onto the vertices w;,w2,...,w, of the polygon with interior angles 
1, b2,--+5 Pn- 


Zo = i. By further requiring z = +o to be mapped onto w = 1, we find 1 = w = expid 
and so ¢ = 0. The required transformation is therefore 

z—i 

z+i 


i 


and is illustrated in figure 24.4. < 


We conclude this section by mentioning the rather curious Schwarz—Christoffel 
transformations Suppose, as shown in figure 24.5, that we are interested in a 
(finite) number of points x1,x2,...,x, on the real axis in the z-plane. Then by 
means of the transformation 


w= {4 fe _ x )O/MNE = x2) 2/91 W(E— Xp) Ga -1 az} +B, (24.30) 
0 


we may map the upper half of the z-plane onto the interior of a closed polygon in 
the w-plane having n vertices w;, W2,...,W, (which are the images of x1, X2,..., Xn) 
with corresponding interior angles 1, ¢2,...,@n, as shown in figure 24.5. The 
real axis in the z-plane is transformed into the boundary of the polygon itself. 
The constants A and B are complex in general and determine the position, 
size and orientation of the polygon. It is clear from (24.30) that dw/dz = 0 at 
X = X1,X2,...,X,, and so the transformation is not conformal at these points. 
There are various subtleties associated with the use of the Schwarz—Christoffel 
transformation. For example, if one of the points on the real axis in the z-plane 
(usually x,) is taken at infinity, then the corresponding factor in (24.30) (i.e. the 
one involving x,,) is not present. In this case, the point(s) x = -too are considered 
as one point, since they transform to a single vertex of the polygon in the w-plane. 


S Strictly speaking, the use of this transformation requires an understanding of complex integrals, 
which are discussed in section 24.8. 
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Figure 24.6 Transforming the upper half of the z-plane into the interior of 
a triangle in the w-plane. 


We can also map the upper half of the z-plane into an infinite open polygon 
by considering it as the limiting case of some closed polygon. 


> Find a transformation that maps the upper half of the z-plane into the triangular region 
shown in figure 24.6 in such a way that the points x, = —1 and x2 = 1 are mapped 
into the points w = —a and w = a, respectively, and the point x3 = +00 is mapped into 
w = ib. Hence find a transformation that maps the upper half of the z-plane into the region 
—a<r<a,s>0 of the w-plane, as shown in figure 24.7. 


Let us denote the angles at w; and w2 in the w-plane by ¢1 = ¢2 = @, where = tan7!(b/a). 
Since x3 is taken at infinity, we may omit the corresponding factor in (24.30) to obtain 


w= afe + 1 P/-"(E = 1)¢/7)-1 as} +B 
0 


= fe — 1)e/-1 as} +B. (24.31) 
0 


The required transformation may then be found by fixing the constants A and B as 
follows. Since the point z = 0 lies on the line segment x,X2, it will be mapped onto the line 
segment w,w 2 in the w-plane, and by symmetry must be mapped onto the point w = 0. 
Thus setting z = 0 and w = 0 in (24.31) we obtain B = 0. An expression for A can be 
found in the form of an integral by setting (for example) z = 1 and w =a in (24.31). 

We may consider the region in the w-plane in figure 24.7 to be the limiting case of the 
triangular region in figure 24.6 with the vertex w3 at infinity. Thus we may use the above, 
but with the angles at w; and w, set to @ = 2/2. From (24.31), we obtain 


wad [| = iasin 
0 f@—1 


By setting z = 1 and w =a, we find iA = 2a/z, so the required transformation is 


2a. 4 
w=—sin‘z.< 
1 
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Figure 24.7 Transforming the upper half of the z-plane into the interior of 
the region —a < r < a, s > 0 in the w-plane. 


24.8 Complex integrals 


Corresponding to integration with respect to a real variable, it is possible to 
define integration with respect to a complex variable between two complex limits. 
Since the z-plane is two-dimensional there is clearly greater freedom and hence 
ambiguity in what is meant by a complex integral. If a complex function f(z) is 
single-valued and continuous in some region R in the complex plane, then we can 
define the complex integral of f(z) between two points A and B along some curve 
in R; its value will depend, in general, upon the path taken between A and B (see 
figure 24.8). However, we will find that for some paths that are different but bear 
a particular relationship to each other the value of the integral does not depend 
upon which of the paths is adopted. 

Let a particular path C be described by a continuous (real) parameter t 
(x <t < f) that gives successive positions on C by means of the equations 


x = x(t), y = y(t), (24.32) 


with t = « and t = f corresponding to the points A and B, respectively. Then the 
integral along path C of a continuous function f(z) is written 


i, f(z) dz (24.33) 
Cc 


and can be given explicitly as a sum of real integrals as follows: 


[ie dz = [uw + iv)(dx + idy) 
Cc c 


= f wax foasifuaysi f vax 

c Cc C Cc 
B B ; B B 

-| Rs dt [es arti f qe arti f Aa dt. 
, at , at , at , at 


(24.34) 
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C3 


Figure 24.8 Some alternative paths for the integral of a function f(z) between 
A and B. 


The question of when such an integral exists will not be pursued, except to state 
that a sufficient condition is that dx/dt and dy/dt are continuous. 


> Evaluate the complex integral of f(z) = z~! along the circle |z| = R, starting and finishing 
at z=R. 


The path C; is parameterised as follows (figure 24.9(a)): 


z(t) = Rceost+iRsint, 0<t<2z, 
whilst f(z) is given by 
1 x—iy 
1O= 5 = ars 
Thus the real and imaginary parts of f(z) are 
x Reost -y Rsint 
age R and v Page Re 


Hence, using expression (24.34), 


1 20 t 20 —sint 
i -—dz -| OST (—R sin t) a | aa Reost dt 
Cc, 7 0 R 0 R 


2n 2n pees 
+if Se Reost dt +i | ( ~) (—Rsint) dt (24.35) 
o oR 0 R 


=0+0+in+in = 271. < 
With a bit of experience, the reader may be able to evaluate integrals like 


the LHS of (24.35) directly without having to write them as four separate real 
integrals. In the present case, 


1 *" _Rsint + iRcost en 
| as -| ane eas e08 a= [ i dt = 2ni. (24.36) 
Cc, 2 9  Rcost+iksint 0 
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Ci 


(a) (b) (c) 


Figure 24.9 Different paths for an integral of f(z) = z~!. See the text for 
details. 


This very important result will be used many times later, and the following should 
be carefully noted: (i) its value, (ii) that this value is independent of R. 

In the above example the contour was closed, and so it began and ended at 
the same point in the Argand diagram. We can evaluate complex integrals along 
open paths in a similar way. 


1 


> Evaluate the complex integral of f(z) =z’ along the following paths (see figure 24.9): 


(i) the contour C, consisting of the semicircle |z| = R in the half-plane y = 0, 
(ii) the contour C; made up of the two straight lines C3q and C3». 


(i) This is just as in the previous example, except that now 0 < t < z. With this change, 
we have from (24.35) or (24.36) that 


oe (24.37) 


CQ 2 
(ii) The straight lines that make up the countour C3; may be parameterised as follows: 


Cras z=(1-—dR+itR forO<t<1; 
Cp, z=-—sR+i(l—s)R for0<s<1l. 


With these parameterisations the required integrals may be written 


[5 cs -{ = = OB agg ‘) eee oe (24.38) 
G R+t(—R+ikR) o IR+s(—R-—iR) 


If we could take over from real-variable theory that, for real t, f(atbty"! dt = b“! In(a+bt) 
even if a and b are complex, then these integrals could be evaluated immediately. However, 
to do this would be presuming to some extent what we wish to show, and so the evaluation 
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must be made in terms of entirely real integrals. For example, the first is given by 


1 _R+iR _ pf l+Id -t=it) 
Ix ————__ dt 


1—t)+itR Jog d—4+e 
'  2r-1 : 1 

= | ——{ att+i |} —~ 4 
i 1—2t+ 20? +if 1—2t+ 20? 

== | In(l—2¢4 27° z 
5 n( + I) + 5 

The second integral on the RHS of (24.38) can also be shown to have the value zi/2. Thus 
dz 


: ria (54)] 
. . 2 : 
“5 E-(B)-$ 
C3 2 


=tTi. <4 


Considering the results of the preceding two examples, which have common 
integrands and limits, some interesting observations are possible. Firstly, the two 
integrals from z = R to z = —R, along C) and C3, respectively, have the same 
value, even though the paths taken are different. It also follows that if we took a 
closed path C4, given by C, from R to —R and C; traversed backwards from —R 
to R, then the integral round C4 of z~! would be zero (both parts contributing 
equal and opposite amounts). This is to be compared with result (24.36), in which 
closed path C;, beginning and ending at the same place as Cy, yields a value 273. 

It is not true, however, that the integrals along the paths C2 and C3 are equal 
for any function f(z), or, indeed, that their values are independent of R in general. 


> Evaluate the complex integral of f(z) = Rez along the paths C,, Cy and C3 shown in 
figure 24.9. 


(i) If we take f(z) = Rez and the contour C, then 

‘ Rez dz= ee Roos t(—Rsint + iRcost) dt = inR?. 
(ii) Using C, as i contour, 

/ Rez dz= if Reost(—Rsint + iRcost) dt = 5inR’. 
(iii) Finally the ‘cteeeal along C3 = C3, + C3, is given by 

| Rez dz = fo —t)R(—R+iR) dt+ [orice — iR) ds 
C3 0 0 
=$R(-1+i)+4R°(1 +i) =iR’. < 
The results of this section demonstrate that the value of an integral between the 
same two points may depend upon the path that is taken between them but, at 


the same time, suggest that, under some circumstances, the value is independent 
of the path. The general situation is summarised in the result of the next section, 
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namely Cauchy’s theorem, which is the cornerstone of the integral calculus of 
complex variables. 

Before discussing Cauchy’s theorem, however, we note an important result 
concerning complex integrals that will be of some use later. Let us consider the 
integral of a function f(z) along some path C. If M is an upper bound on the 
value of |f(z)| on the path, Le. |f(z)| <M on C, and L is the length of the path C, 


then 
| i. f(z) dz 
Cc 


It is straightforward to verify that this result does indeed hold for the complex 
integrals considered earlier in this section. 


< [veria < m [a = ML. (24.39) 


24.9 Cauchy’s theorem 


Cauchy’s theorem states that if f(z) is an analytic function, and f’(z) is continuous 
at each point within and on a closed contour C, then 


f f(z) dz =0. (24.40) 
Cc 


In this statement and from now on we denote an integral around a closed contour 
by ¢.. 

To prove this theorem we will need the two-dimensional form of the divergence 
theorem, known as Green’s theorem in a plane (see section 11.3). This says that 
if p and q are two functions with continuous first derivatives within and on a 
closed contour C (bounding a domain R) in the xy-plane, then 


II e x) aN fo ayaa): (24.41) 


With f(z) =u+iv and dz = dx +i dy, this can be applied to 


t= $ fe) de= pu de—o dy) +i p (de tu dy) 


r= ff i. ae at dx dy +i ff & ee +3 dx dy. (24.42) 


Now, recalling that f(z) is analytic and therefore that the Cauchy—Riemann 
relations (24.5) apply, we see that each integrand is identically zero and thus J is 
also zero; this proves Cauchy’s theorem. 

In fact, the conditions of the above proof are more stringent than they need 
be. The continuity of f’(z) is not necessary for the proof of Cauchy’s theorem, 


to give 
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C, 


Figure 24.10 Two paths C; and C) enclosing a region R. 


analyticity of f(z) within and on C being sufficient. However, the proof then 
becomes more complicated and is too long to be given heres 

The connection between Cauchy’s theorem and the zero value of the integral 
of z~! around the composite path C4 discussed towards the end of the previous 
section is apparent: the function z~! is analytic in the two regions of the z-plane 
enclosed by contours (C2 and C3q) and (C2 and C3,). 


> Suppose two points A and B in the complex plane are joined by two different paths C, 
and C,. Show that if f(z) is an analytic function on each path and in the region enclosed 
by the two paths, then the integral of f(z) is the same along C; and C). 


The situation is shown in figure 24.10. Since f(z) is analytic in R, it follows from Cauchy’s 
theorem that we have 


f(z)az— | fi) az= $ fe) dz =0, 
lon on Ci=Cy 


since C; — C, forms a closed contour enclosing R. Thus we immediately obtain 


[, f(z) dz = a f(z) az, 


and so the values of the integrals along C; and C) are equal. < 


An important application of Cauchy’s theorem is in proving that, in some 
cases, it is possible to deform a closed contour C into another contour y in such 
a way that the integrals of a function f(z) around each of the contours have the 
same value. 


8 The reader may refer to almost any book that is devoted to complex variables and the theory of 
functions. 


850 


24.10 CAUCHY’S INTEGRAL FORMULA 


Figure 24.11 The contour used to prove the result (24.43). 


> Consider two closed contours C and y in the Argand diagram, y being sufficiently small 
that it lies completely within C. Show that if the function f(z) is analytic in the region 
between the two contours then 


¢ f(z) dz = ¢ f(z) dz. (24.43) 
ic oY 


To prove this result we consider a contour as shown in figure 24.11. The two close parallel 
lines C; and C, join y and C, which are ‘cut’ to accommodate them. The new contour I 
so formed consists of C, Ci, y and Cp. 

Within the area bounded by I, the function f(z) is analytic, and therefore, by Cauchy’s 
theorem (24.40), 


$ f(z) dz =0. (24.44) 
T 


Now the parts C; and C, of I are traversed in opposite directions, and in the limit lie on 
top of each other, and so their contributions to (24.44) cancel. Thus 


$ f(z) dz+ $ f(z) dz =0. (24.45) 
Cc y 


The sense of the integral round y is opposite to the conventional (anticlockwise) one, and 
so by traversing y in the usual sense, we establish the result (24.43). < 


A sort of converse of Cauchy’s theorem is known as Morera’s theorem, which 
states that if f(z) is a continuous function of z in a closed domain R bounded by 
a curve C and, further, fc f(z) dz =0, then f(z) is analytic in R. 


24.10 Cauchy’s integral formula 


Another very important theorem in the theory of complex variables is Cauchy’s 
integral formula, which states that if f(z) is analytic within and on a closed 
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contour C and zp is a point within C then 


1 f 10 4 


2ni Jc Zz — 20 


f (Zo) (24.46) 


This formula is saying that the value of an analytic function anywhere inside a 
closed contour is uniquely determined by its values on the contour’ and that the 
specific expression (24.46) can be given for the value at the interior point. 

We may prove Cauchy’s integral formula by using (24.43) and taking y to be a 
circle centred on the point z = zo, of small enough radius p that it all lies inside 
C. Then, since f(z) is analytic inside C, the integrand f(z)/(z — zo) is analytic 
in the space between C and y. Thus, from (24.43), the integral around y has the 
same value as that around C. 

We then use the fact that any point z on y is given by z = zo + pexpi0 (and 
so dz =ipexpi0 d0). Thus the value of the integral around y is given by 

pa Gf SQ. ga [7 feat exp i®), wie de 
» 2— 20 0 pexpi0 
2n 
= f(zo + pexpid) dé. 
0 


If the radius of the circle y is now shrunk to zero, i.e. p > 0, then I — 2zif (zo), 
thus establishing the result (24.46). 

An extension to Cauchy’s integral formula can be made, yielding an integral 
expression for f’(zo): 


f'(zo) = ce : mee (24.47) 
GC 


~ Ini Je (z — 20)? 


under the same conditions as previously stated. 


> Prove Cauchy’s integral formula for f'(zo) given in (24.47). 


To show this, we use the definition of a derivative and (24.46) itself to evaluate 


7.) — yi. fo +h) — f(Z0) 
od a 


3 1 f(z) 1 1 
tim [5 h Z—-Z2—-h z-—2% a 


an 1 f(z) 
7 0 laf (z — 29 — h)(z — 20) | 
1 f(z) 


~ 2ni Jo (@ — 20) 
which establishes result (24.47). < 


2 > 


S The similarity between this and the uniqueness theorem for the Laplace equation with Dirichlet 
boundary conditions (see chapter 20) is apparent. 
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Further, it may be proved by induction that the nth derivative of f(z) is also 
given by a Cauchy integral, 


(n) _ al f(z) dz 
f'"(Z0) = 4 Gaye (24.48) 


c (z—20 
Thus, if the value of the analytic function is known on C then not only may the 
value of the function at any interior point be calculated, but also the values of 
all its derivatives. 
The observant reader will notice that (24.48) may also be obtained by the 
formal device of differentiating under the integral sign with respect to zo in 
Cauchy’s integral formula (24.46): 


1 
f(z) = 


f(z) 
abe azn 24 © 


a! f(z) dz 


Oni c(zZ— zor! : 


> Suppose that f(z) is analytic inside and on a circle C of radius R centred on the point 
Zz = 2. If |f(z)| < M on the circle, where M is some constant, show that 


Mn! 
IP (eo)l < SE. 


(24.49) 


From (24.48) we have 


n! f(z) dz 
(n) = fe a eh Pei 
[pr Go) = - (2 — zo)" 
and on using (24.39) this becomes 
M Mn! 
n) Sedge a 
if' (2 0) = = Ret Bt 27R = R- 


This result is known as Cauchy’s inequality. < 


We may use Cauchy’s inequality to prove Liouville’s theorem, which states that 
if f(z) is analytic and bounded for all z then f is a constant. Setting n = 1 in 
(24.49) and letting R > oo, we find |f’(zo)| = 0 and hence f’(zo) = 0. Since f(z) is 
analytic for all z, we may take zp as any point in the z-plane and thus f’(z) =0 
for all z; this implies f(z) = constant. Liouville’s theorem may be used in turn to 
prove the fundamental theorem of algebra (see exercise 24.9). 


24.11 Taylor and Laurent series 


Following on from (24.48), we may establish Taylor’s theorem for functions of a 
complex variable. If f(z) is analytic inside and on a circle C of radius R centred 
on the point z = Zo, and z is a point inside C, then 


f(z) = - An(z — zo)", (24.50) 
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where a, is given by f(zo)/n!. The Taylor expansion is valid inside the region 
of analyticity and, for any particular zo, can be shown to be unique. 

To prove Taylor’s theorem (24.50), we note that, since f(z) is analytic inside 
and on C, we may use Cauchy’s formula to write f(z) as 


F(9) 
coz 


where € lies on C. Now we may expand the factor (€ —z)~ 


in (z — zo)/(€ — Zo), 
Ls il = zZ—z\" 
E=Z a, ; 


1 
fZ= oni dé, (24.51) 


' as a geometric series 


o (24.51) becomes 


f(E) z— 20 2 
f@ d=s¢ H te). & 


10) 
allie n LO) Z 
ni Le a i ay 
j ee Anif™ 
= 55 oe - 20)" ua (24.52) 
n=0 ; 


where we have used Cauchy’s integral formula (24.48) for the derivatives of 
f(z). Cancelling the factors of 2zi, we thus establish the result (24.50) with 


Gn = f(z0)/n!. 


> Show that if f(z) and g(z) are analytic in some region R, and f(z) = g(z) within some 
subregion S of R, then f(z) = g(z) throughout R. 


It is simpler to consider the (analytic) function h(z) = f(z)—g(z), and to show that because 
h(z) = 0 in S it follows that h(z) = 0 throughout R. 
If we choose a point z = zo in S, then we can expand h(z) in a Taylor series about Zo, 


h(z) = h(zo) + h'(zo)(z — 20) + 3h" (zolz — 20)” +++ 5 


which will converge inside some circle C that extends at least as far as the nearest part of 
the boundary of R, since h(z) is analytic in R. But since zo lies in S, we have 


h(zo) = h'(zo) = h'"(zo) = +++ = 0, 


and so h(z) = 0 inside C. We may now expand about a new point, which can lie anywhere 
within C, and repeat the process. By continuing this procedure we may show that h(z) = 0 
throughout R. 

This result is called the identity theorem and, in fact, the equality of f(z) and g(z) 
throughout R follows from their equality along any curve of non-zero length in R, or even 
at a countably infinite number of points in R. 4 


So far we have assumed that f(z) is analytic inside and on the (circular) 
contour C. If, however, f(z) has a singularity inside C at the point z = Zo, then it 
cannot be expanded in a Taylor series. Nevertheless, suppose that f(z) has a pole 
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of order p at z = zo but is analytic at every other point inside and on C. Then 
the function g(z) = (z — zo)’ f(z) is analytic at z = zo, and so may be expanded 
as a Taylor series about z = zo: 


g(z) = Ss bn(z — zo)". (24.53) 


n=0 


Thus, for all z inside C, f(z) will have a power series representation of the form 
a_ a_ 
f(2) = ea ee + ay + a(z — 20) + a9(z — 20) +-°-, 

° ° (24.54) 


with a_, # 0. Such a series, which is an extension of the Taylor expansion, is 
called a Laurent series. By comparing the coefficients in (24.53) and (24.54), we 
see that a, = by+p. Now, the coefficients b, in the Taylor expansion of g(z) are 
seen from (24.52) to be given by 


_ 2%) 1 fee) 
n! Qni J (z—zp)tt! 


n 


and so for the coefficients a, in (24.54) we have 


1 g(z) pede (2) 


ah = —  — az = — § ————_ ez 
"Ini J (z —zo)rt le Qni J (z—zp)rt 


an expression that is valid for both positive and negative n. 

The terms in the Laurent series with n > 0 are collectively called the analytic 
part, whilst the remainder of the series, consisting of terms in inverse powers of 
Z — Zo, is called the principal part. Depending on the nature of the point z = zo, 
the principal part may contain an infinite number of terms, so that 

+to0 
f(z) = > an(z — 20)". (24.55) 
n>=—0o0 

In this case we would expect the principal part to converge only for |(z — z)~|| 
less than some constant, ie. outside some circle centred on zo. However, the 
analytic part will converge inside some (different) circle also centred on zo. If the 
latter circle has the greater radius then the Laurent series will converge in the 
region R between the two circles (see figure 24.12); otherwise it does not converge 
at all. 

In fact, it may be shown that any function f(z) that is analytic in a region 
R between two such circles C; and C2 centred on z = zo can be expressed as 
a Laurent series about zo that converges in R. We note that, depending on the 
nature of the point z = zo, the inner circle may be a point (when the principal 
part contains only a finite number of terms) and the outer circle may have an 
infinite radius. 

We may use the Laurent series of a function f(z) about any point z = zo to 
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Figure 24.12 The region of convergence R for a Laurent series of f(z) about 
a point z = Zo where f(z) has a singularity. 


classify the nature of that point. If f(z) is actually analytic at z = zo, then in 
(24.55) all a, for n < 0 must be zero. It may happen that not only are all a, 
zero for n < 0 but ao, aj, ..., Gm—1 are all zero as well. In this case, the first 
non-vanishing term in (24.55) is ay,(z — zo)”, with m > 0, and f(z) is then said to 
have a zero of order m at z = Zo. 

If f(z) is not analytic at z = zo, then two cases arise, as discussed above (p is 
here taken as positive): 


(i) it is possible to find an integer p such that a_p #0 but a_p~ =0 for all 
integers k > 0; 


(ii) it is not possible to find such a lowest value of —p. 


In case (i), f(z) is of the form (24.54) and is described as having a pole of order 
p at z = zo; the value of a_; (not a_,) is called the residue of f(z) at the pole 
z = Zo, and will play an important part in later applications. 

For case (ii), in which the negatively decreasing powers of z — zp do not 
terminate, f(z) is said to have an essential singularity. These definitions should be 
compared with those given in section 24.6. 


> Find the Laurent series of 
1 


I@)= ay 


about the singularities z = 0 and z = 2 (separately). Hence verify that z = 0 is a pole of 
order 1 and z = 2 is a pole of order 3, and find the residue of f(z) at each pole. 


To obtain the Laurent series about z = 0, we make the factor in parentheses in the 
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denominator take the form (1 — «z), where « is some constant, and thus obtain 


1 
I@Q=—-F aap 


1 3 32 = 52? 
8z 16 16 32 
Since the lowest power of z is —1, the point z = 0 is a pole of order 1. The residue of f(z) 
at z = 0 is simply the coefficient of z~! in the Laurent expansion about that point and is 
equal to —1/8. 
The Laurent series about z = 2 is most easily found by letting z = 2+ é (or z—2=€) 
and substituting into the expression for f(z) to obtain 
1 1 
Q+58 6), +€/2) 


/-@-0)-0-0~4 


283 pte 16 32 
1 1 1 1 2-2 


“3G-p) 42-27 'Be—-y 16 32 


From this series we see that z = 2 is a pole of order 3 and that the residue of f(z) at z = 2 
is 1/8. < 


faa 


As we shall see in the next few sections, finding the residue of a function 
at a singularity is of crucial importance in the evaluation of complex integrals. 
Specifically, formulae exist for calculating the residue of a function at a particular 
(singular) point z = zo without having to expand the function explicitly as a 
Laurent series about zo and identify the coefficient of (z — z9)~!. The type of 
formula generally depends on the nature of the singularity at which the residue 
is required. 


> Suppose that f(z) has a pole of order m at the point z = zo. By considering the Laurent 
series of {(z) about zo, derive a general expression for the residue R(zZo) of f(z) at z = Zo. 
Hence evaluate the residue of the function 


expiz 
@ + 1p 


at the point z =i. 


If f(z) has a pole of order m at z = Zo, then its Laurent series about this point has the 
form 

=i a_ 

bees 4 $ ay + ay(z — 20) + an(z — 20) $20, 

(z — 20)” (z — 20) 

which, on multiplying both sides of the equation by (z — zo)", gives 


(2 — 20)" (2) = Gon + G@mai(Z — 20) +++ + aa(z = 20)" | Fee, 


F(z) 
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Differentiating both sides m— 1 times, we obtain 


m—L 


dzm-! 


[(e — 20)"f(2)] = (m= 1)! a +> blz — 20)" 


n=1 


for some coefficients b,. In the limit z — zo, however, the terms in the sum disappear, and 
after rearranging we obtain the formula 


; 1a 
Rezo) = 4-1 = im { (e—ay"sen}. (24.56) 


z—29 (m = 1)! dzm-! 


which gives the value of the residue of f(z) at the point z = zo. 
If we now consider the function 


_ expiz _ expiz 
I)= Grey ~ @ +e 
we see immediately that it has poles of order 2 (double poles) at z =i and z = —i. To 


calculate the residue at (for example) z = i, we may apply the formula (24.56) with m = 2. 
Performing the required differentiation, we obtain 


d ; d | expiz 
<le- 7) 


dz |(z +i) 
= ale + i)iexpiz — 2(expiz)(z +i). 
Setting z =i, we find the residue is given by 
; 1 1 ti aS i 
R(i) Tlie (—4ie 4ie') x * 


An important special case of (24.56) occurs when f(z) has a simple pole (a pole 
of order 1) at z = zo. Then the residue at zo is given by 


R(Zo) = jim [(z — 20) f(z). (24.57) 
If f(z) has a simple pole at z = zo and, as is often the case, has the form 


g(z)/h(z), where g(z) is analytic and non-zero at zp and h(zo) = 0, then (24.57) 
becomes 


R(co) = lim ( ae = g(20) lim ( — 
= g(Zo) lim ae ta) (24.58) 


z20 W'(z) (zo) 


where we have used l’H6pital’s rule. This result often provides the simplest way 
of determining the residue at a simple pole. 


24.12 Residue theorem 


Having seen from Cauchy’s theorem that the value of an integral round a closed 
contour C is zero if the integrand is analytic inside the contour, it is natural to 
ask what value it takes when the integrand is not analytic inside C. The answer 
to this is contained in the residue theorem, which we now discuss. 
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Suppose the function f(z) has a pole of order m at the point z = zo, and so 
can be written as a Laurent series about zo of the form 
foe) 
f(z) = S> ay(z - 20)". (24,59) 
n=—m 
Now consider the integral J of f(z) around a closed contour C that encloses 
Z = Zo, but no other singular points. Using Cauchy’s theorem, this integral has 
the same value as the integral around a circle y of radius p centred on z = Zo, 
since f(z) is analytic in the region between C and ;. On the circle we have 
z =z) + pexpid (and dz = ipexpi@ d@), and so 


T= a) dz 


Il 
Pe] 
Se 
N 
| 
a 

& 


ll 


2n 
os an | ip"* expfi(n + 1)6] dd. 


n=—m 


For every term in the series with n 4 —1, we have 


ip"*! exp[i(n + 1)6] 7m 


=0, 
i(n + 1) 0 


2n 
| ip"*! exp[i(n + 1)6] dO = | 
0 


but for the n = —1 term we obtain 


2n 
i i dO = 2ni. 
0 


Therefore only the term in (z — zo)! contributes to the value of the integral 
around y (and therefore C), and I takes the value 


I= ¢ f(z) dz = 2mia_y. (24.60) 


Thus the integral around any closed contour containing a single pole of general 
order m (or, by extension, an essential singularity) is equal to 27i times the residue 
of f(z) at z = Zo. 

If we extend the above argument to the case where f(z) is continuous within 
and on a closed contour C and analytic, except for a finite number of poles, 
within C, then we arrive at the residue theorem 


‘(z) dz = 2ni j 24.61 
¢ fe) Zz He ( ) 


where »; R; is the sum of the residues of f(z) at its poles within C. 
The method of proof is indicated by figure 24.13, in which (a) shows the original 
contour C referred to in (24.61) and (b) shows a contour C’ giving the same value 
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(a) (b) 


Figure 24.13 The contours used to prove the residue theorem: (a) the original 
contour; (b) the contracted contour encircling each of the poles. 


to the integral, because f is analytic between C and C’. Now the contribution to 
the C’ integral from the polygon (a triangle for the case illustrated) joining the 
small circles is zero, since f is also analytic inside C’. Hence the whole value of 
the integral comes from the circles and, by result (24.60), each of these contributes 
2ni times the residue at the pole it encloses. All the circles are traversed in their 
positive sense if C is thus traversed and so the residue theorem follows. Formally, 
Cauchy’s theorem (24.40) is a particular case of (24.61) in which C encloses no 
poles. 

Finally we prove another important result, for later use. Suppose that f(z) has 
a simple pole at z = z) and so may be expanded as the Laurent series 


f(z) = $(z) + a-A(z — 20), 


where ¢(z) is analytic within some neighbourhood surrounding zo. We wish to 
find an expression for the integral J of f(z) along an open contour C, which is 
the arc of a circle of radius p centred on z = zo given by 


Iz—zol =p, 0) <arg(z — 2) < >, (24.62) 


where p is chosen small enough that no singularity of f, other than z = zo, lies 
within the circle. Then J is given by 


I= [ie dz= i o(z) dz +a_4 fe —z9)! dz. 


If the radius of the arc C is now allowed to tend to zero, then the first integral 
tends to zero, since the path becomes of zero length and ¢ is analytic and 
therefore continuous along it. On C, z = pe! and hence the required expression 
for I is 


05 
I=lim | f(z) dz =lim ( i _ ine” i) = ia_\(0.—6;). (24.63) 
Cc p0 0 pe' 


p70 
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We note that result (24.60) is a special case of (24.63) in which 02 is equal to 
0; +27. 


24.13 Definite integrals using contour integration 


The remainder of this chapter is devoted to methods of applying contour integra- 
tion and the residue theorem to various types of definite integral. However, three 
applications of contour integration, in which obtaining a value for the integral is 
not the prime purpose of the exercise, have been postponed until chapter 25. They 
are the location of the zeros of a complex polynomial, the evaluation of the sums 
of certain infinite series and the determination of inverse Laplace transforms. 

For the integral evalations considered here, not much preamble is given since, 
for this material, the simplest explanation is felt to be via a series of worked 
examples that can be used as models. 


24.13.1 Integrals of sinusoidal functions 


Suppose that an integral of the form 
2n 
i F(cos 6, sin 0) dO (24.64) 
0 


is to be evaluated. It can be made into a contour integral around the unit circle 
C by writing z = expi0, and hence 
cos 0 = $(z +27"), sin@ = —ti(z —z7'), d0 = —iz! dz. (24.65) 


This contour integral can then be evaluated using the residue theorem, provided 
the transformed integrand has only a finite number of poles inside the unit circle 
and none on it. 


> Evaluate 


ue cos 20 
i ———_—— 40 b 0. 
i a@+b2—2abcosd ” eis 


By de Moivre’s theorem (section 3.4), 
cos nO = $(2" +27"). (24.67) 


Using n = 2 in (24.67) and straightforward substitution for the other functions of @ in 
(24.66) gives 


i zat 
.= 5 é PEaaheana 


Thus there are two poles inside C, a double pole at z = 0 and a simple pole at z = a/b 
(recall that b > a). 

We could find the residue of the integrand at z = 0 by expanding the integrand as a 
Laurent series in z and identifying the coefficient of z~!. Alternatively, we may use the 
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formula (24.56) with m = 2. Choosing the latter method and denoting the integrand by 
f(z), we have 


d d z+ 
a fell a (@ —a/byz —b/a) 
_ (g—a/b)(z — b/a)4z? — (24 + I)[(z — a/b) + (2 — b/a) 
(2 — a/b)*(z — b/ay : 
Now setting z = 0 and applying (24.56), we find 


ab 
R(O) = b + a 
For the simple pole at z = a/b, equation (24.57) gives the residue as 
: a/b) +1 
Rla/b) = tim, [2 — a/bf(@)] = AE 
_ at+ot 
ab(b? — a?) 
Therefore by the residue theorem 
i fae@t+bh at + bt 2na* 


I =22i x —— | ——— - ————_| = =>. < 
Uap ll ak aE =P) || > BB eS oa) 


24.13.2 Some infinite integrals 


We next consider the evaluation of an integral of the form 


ee) 


. f(x) dx, 


0 


where f(z) has the following properties: 


(i) f(z) is analytic in the upper half-plane, Im z > 0, except for a finite number 
of poles, none of which is on the real axis; 

(ii) on a semicircle T° of radius R (figure 24.14), R times the maximum of |f| 
on I’ tends to zero as R > ~ (a sufficient condition is that zf(z) — 0 as 
[Z| — 00); 


(iii) [°) f(x) dx and J.” f(x) dx both exist. 


[re dz 


condition (ii) ensures that the integral along I tends to zero as R > «, after 
which it is obvious from the residue theorem that the required integral is given 
by 


Since 


< 2xR x (maximum of |f| on T), 


: f(x) dx = 2ni x (sum of the residues at poles with Imz > 0). 
ms (24.68) 
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Figure 24.14 A semicircular contour in the upper half-plane. 


> Evaluate 


ie d. 
I -| mae where a is real. 
0 


The complex function (z* + a?)~* has poles of order 4 at z = -tai, of which only z = ai 
is in the upper half-plane. Conditions (ii) and (iii) are clearly satisfied. For higher-order 
poles, formula (24.56) for evaluating residues can be tiresome to apply. So, instead, we put 
z =ai+€ and expand for small € to obtain’ 


1 1 1 ié\~ 
ray Qaie+ey aie” 2a) 
The coefficient of €~! is given by 


1 (—4)(—5)(-6) (/-i\? 5i 
(2a)4 3] € ) 


and hence by the residue theorem 


. dx _ 102 
4 02 +a2)4 ~~ 32a7’ 
and so I = 52/(32a’). < 


Condition (i) of the previous method required there to be no poles of the 
integrand on the real axis, but in fact simple poles on the real axis can be 
accommodated by indenting the contour as shown in figure 24.15. The indentation 
at the pole z = zp is in the form of a semicircle y of radius p in the upper half- 
plane, thus excluding the pole from the interior of the contour. 


S This illustrates another useful technique for determining residues. 
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Figure 24.15 An indented contour used when the integrand has a simple 
pole on the real axis. 


What is then obtained from a contour integration, apart from the contributions 
for TP’ and y, is called the principal value of the integral, defined as p — 0 by 


P / : iS tes / TP Biie  Gies 


—R Zo+p 
The remainder of the calculation goes through as before, but the contribution 
from the semicircle, y, must be included. Result (24.63) of section 24.12 shows 
that since only a simple pole is involved its contribution is 


“ialae, (24.69) 


where a_; is the residue at the pole and the minus sign arises because y is 
traversed in the clockwise (negative) sense. 

We defer giving an example of an indented contour until we have established 
Jordan’s lemma; we will then work through an example illustrating both. Jordan’s 
lemma enables infinite integrals involving sinusoidal functions to be evaluated. 


For a function f(z) of a complex variable z, if 
(i) f(z) is analytic in the upper half-plane except for a finite number of poles in Imz > 0, 
(ii) the maximum of |f(z)| > 0 as |z| > co in the upper half-plane, 
(iii) m>0, 
then 


Ip = | e™ f(z)dz 30 asR>o, (24.70) 
i 
where I is the same semicircular contour as in figure 24.14. 


Note that this condition (ii) is less stringent than the earlier condition (ii) (see the 
start of this section), since we now only require M(R) > 0 and not RM(R) — 0, 
where M is the maximum! of |f(z)| on |z| = R. 


S More strictly, the least upper bound. 
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The proof of the lemma is straightforward once it has been observed that, for 
0<0<7/2, 


altNw 


in0 
1> — = (24.71) 


Then, since on I’ we have | exp(imz)| = | exp(—mR sin 6), 
; Tt . n/2 : 
Ir< | le” f(z)| |\dz|< MR | emRsin? 49 —2MR | emRsin® gg. 
r 0 0 
Thus, using (24.71), 


n/2 
Ir <2MR i, emR(20/n) gg — 2M (1-e™’) < nee 
0 m 


hence, as R — oo, Ir tends to zero since M tends to zero. 


> Find the principal value of 


00. 
cos mx 
i — dx, for a real, m> 0. 


o X—a 


Consider the function (z — a)~' exp(imz); although it has no poles in the upper half-plane 
it does have a simple pole at z = a, and further |(z — a)~'| > 0 as |z| > oo. We will use a 
contour like that shown in figure 24.15 and apply the residue theorem. Symbolically, 


i +f a je i | =e (24.72) 


Now as R > and p > 0 we have {, — 0, by Jordan’s lemma, and from (24.68) and 
(24.69) we obtain 


P / o dx—ina_, =0, (24.73) 
~»X—a 


where a_, is the residue of (z — a)~! exp(imz) at z = a, which is exp(ima). Then taking the 
real and imaginary parts of (24.73) gives 


” cosmx ; ; 
P —— dx = —nsinma, as required, 
-o X—a 
oan: 
sin mx 
P i dx = 1 cosma, as a bonus. < 
Xa 
—o 


24.13.3 Integrals of multivalued functions 


We have discussed briefly some of the properties and difficulties associated with 
certain multivalued functions such as z!/? or Lnz. It was mentioned that one 
method of managing such functions is by means of a ‘cut plane’. A similar 
technique can be used with advantage to evaluate some kinds of infinite integral 
involving real functions for which the corresponding complex functions are multi- 
valued. A typical contour employed for functions with a single branch point 


865 


COMPLEX VARIABLES 


5 
"a 


Figure 24.16 A typical cut-plane contour for use with multivalued functions 
that have a single branch point located at the origin. 


located at the origin is shown in figure 24.16. Here I is a large circle of radius R 
and y is a small one of radius p, both centred on the origin. Eventually we will 
let R-> © and p— 0. 

The success of the method is due to the fact that because the integrand is 
multivalued, its values along the two lines AB and CD joining z = p toz =R 
are not equal and opposite although both are related to the corresponding real 
integral. Again an example provides the best explanation. 


> Evaluate 


ee dx 
rf Ea: a>o0. 


We consider the integrand f(z) = (z +.a)~*z~!/? and note that |zf(z)| > 0 on the two 
circles as p > 0 and R > o. Thus the two circles make no contribution to the contour 


integral. 
The only pole of the integrand inside the contour is at z = —a (and is of order 3). 
To determine its residue we put z = —a+é and expand (noting that (—a)!/? equals 


a’? exp(in/2) = ia'/?): 
1 1 


(@tapz? ~ Bia — Ea? 
1 1é 32 
= ge (14524554). 


The residue is thus —3i/(8a°/*). 
The residue theorem (24.61) now gives 


i +f+f +f =2ni = 
AB r DC fo 8ar/? } 
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We have seen that f, and f' vanish, and if we denote z by x along the line AB then it 
has the value z = xexp 27i along the line DC (note that exp 2zi must not be set equal to 
1 until after the substitution for z has been made in [,,.). Substituting these expressions, 


fe dx is dx 3n 
9 (x+a)3x!/? « [xexp2ni+a]x!/2exp(42ni) — 4a*/?” 


Thus 
1 1 is dx _ 3n 
expni) Jy (x+ax'/? ~~ 45/2 
and 
1 3x 
I= 3 x 4g < 


Several other examples of integrals of multivalued functions around a variety 
of contours are included in the exercises that follow. 


24.14 Exercises 
24.1 Find an analytic function of z = x + iy whose imaginary part is 
(y cosy +xsin y)exp x. 
24.2 Find a function f(z), analytic in a suitable part of the Argand diagram, for which 
sin 2x 
Re f = ——————_.. 
as cosh 2y — cos 2x 


Where are the singularities of f(z)? 
24.3 Find the radii of convergence of the following Taylor series: 


oa) 


“2 nz" 
(a) Tan’ (b) pan re 


n=2 n=1 


fee) ioe} Ta 
(c) >: zn" (d) a (7 +r) z", with p real. 


n=1 n=1 


24.4 Find the Taylor series expansion about the origin of the function f(z) defined by 


where p is a constant. Hence verify that f(z) is a convergent series for all z. 


24.5 Determine the types of singularities (if any) possessed by the following functions 
atz=Oandz=o: 
(a) (2-2), (b) (L+2°)/2?, (c) sinh(1/z), 
(d) e/2°, (e) 27/1 +27). 
24.6 Identify the zeros, poles and essential singularities of the following functions: 
(a) tanz, (b) [(z — 2)/z7] sin[1/(1—z)], (c) exp(1/z), 


(d) tan(1/z), (e) 27”. 
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24.7 


24.8 


24.9 


24.10 


24.11 


24.12 


24.13 


Find the real and imaginary parts of the functions (i) z’, (ii) e?, and (iii) cosh zz. 
By considering the values taken by these parts on the boundaries of the region 
0 < x, y < 1, determine the solution of Laplace’s equation in that region that 
satisfies the boundary conditions 


$(x,0) = 0, $(0, y) = 0, 
P(x, 1) = x, PL y) =y +sin zy. 
Show that the transformation 


fe 1 
? =f Gon & 


transforms the upper half-plane into the interior of a square that has one corner 
at the origin of the w-plane and sides of length L, where 


n/2 
= | cosec!/? @ dd. 
0 


The fundamental theorem of algebra states that, for a complex polynomial p,(z) 
of degree n, the equation p,(z) = 0 has precisely n complex roots. By applying 
Liouville’s theorem (see the end of section 24.10) to f(z) = 1/p,(z), prove that 
P»(z) = 0 has at least one complex root. Factor out that root to obtain p,_1(z) 
and, by repeating the process, prove the above theorem. 

Show that, if a is a positive real constant, the function exp(iaz’) is analytic and 
— 0 as |z| > oo for 0 < argz < 1/4. By applying Cauchy’s theorem to a suitable 


contour prove that 
: 7 
| cos(ax?) dx = ,/—. 
0 8a 


The function 

fl) =(1=23)'” 
of the complex variable z is defined to be real and positive on the real axis in 
the range —1 < x < 1. Using cuts running along the real axis for 1 < x < +00 
and —o < x < —1, show how f(z) is made single-valued and evaluate it on the 


upper and lower sides of both cuts. 
Use these results and a suitable contour in the complex z-plane to evaluate the 


integral 
= dx 
dy x0 = D2" 


Confirm your answer by making the substitution x = sec 0. 
By considering the real part of 


—iz"! dz 
/ Tag +2) +a?’ 
where z = expi@ and n is a non-negative integer, evaluate 
" cos nO 
| aoe 
for a real and > 1. 
Prove that if f(z) has a simple zero at zo, then 1/f(z) has residue 1/f’(zo) there. 


Hence evaluate 
7 sind 
, — dd, 
_, a— sind 


868 


where a is real and > 1. 


24.14 EXERCISES 


24.14 


24.15 


24.16 


24.17 


24.18 


24.19 


24.20 


24.21 


Prove that, for « > 0, the integral 


” tsinat 
—, dt 
| 1+? 
has the value (2/2) exp(—«). 
Prove that 


cos mx T 
dx = — (4, —m/2 __ ,—m fi > 0. 
i baseai & al e e”) orm 


Show that the principal value of the integral 
” cos(x/a) 
is —(z/a) sin 1. 


The following is an alternative (and roundabout!) way of evaluating the Gaussian 
integral. 


(a) Prove that the integral of [exp(izz?)]cosec zz around the parallelogram with 
corners +1/2 + Rexp(iz/4) has the value 2i. 

(b) Show that the parts of the contour parallel to the real axis do not contribute 
when R > . 


(c) Evaluate the integrals along the other two sides by putting z’ = r exp(iz/4) 


and working in terms of z’ + 5 and z’ — ; Hence, by letting R — oo show 


that 
a em dr = 1. 


By applying the residue theorem around a wedge-shaped contour of angle 2z/n, 
with one side along the real axis, prove that the integral 


a dx 
o 1+x" 


where n is real and > 2, has the value (z/n)cosec(z/n). 
Using a suitable cut plane, prove that if « is real and 0 < « < 1 then 


ao xo 
| — dx 
9 L+x 
has the value mcosec ma. 


Show that 
ad Inx > 


By integrating a suitable function around a large semicircle in the upper half- 
plane and a small semicircle centred on the origin, determine the value of 


= FP nx? 
r= tase 


and deduce, as a by-product of your calculation, that 


”  Inx 
[ faa’ 
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24.22 


The equation of an ellipse in plane polar coordinates r, 0, with one of its foci at 
the origin, is 
f = 1—ecos0, 
7 
where ! is a length (that of the latus rectum) and e (0 < € < 1) is the eccentricity 
of the ellipse. Express the area of the ellipse as an integral around the unit circle 
in the complex plane, and show that the only angularly of the integrand inside 
the circle is a double pole at z) = e~! — (e~* — 1)!” 
By setting z = z+ € and oe ine the integrand i in powers of €, find the 
residue at zo and hence show that the area is equal to 2/?(1 — e?)~>”. 
[In terms of the semi-axes a and b of the ellipse, | = b?/a and e? = (a’—b’)/a’.] 


24.15 Hints and answers 


du/dy = —(expx)(ycosy + xsiny + siny); zexpz. 

(a) 1; (b) 1; (¢) 1; (d) e®. 

(a) Analytic, analytic; (b) double pole, single pole; (c) essential singularity, ana- 

lytic; (d) triple pole, essential singularity; (e) branch point, branch point. 

(i) x? — y?, 2xy; (ii) e* cos y, e* sin y; (iii) cosh zx cos zy, sinh zx sin zy; 

(x, y) = xy + (sinh zx sin zy)/ sinh z. 

Assume that p,(x) (r =n,n—1,...,1) has no roots and then argue by the method 

of contradiction. 

With 0 < 0, < 2m and —n < 02 < a, f(z) = (rr2)'/? exp[i(0, + 02 — 2) ]. The four 

values are +i(x? — 1)'/*, with the plus sign corresponding to points near the cut 

that lie in the second and fourth quadrants. I = 2/2. 

The only pole inside the unit circle is at z = ia — i(a* — 1)'/”; the residue is given 
—(i/2)(a? —1)~'”*; the integral has value 2x[a(a? — 1)~'/? — 1]. 

Factorise the denominator, showing that the relevant simple poles are at i/2 and 

i, 

(a) The only pole is at the origin with residue x7!; 

(b) each is OL[exp(—aR? $ /22R)] 

(c) the sum of the integrals is 2i ee exp(—zr?) dr. 

Use a contour like that shown in figure 24.16. 

Note that pln" p — 0 as p > 0 for all n. When z is on the negative real axis, 

(Inz)* contains three terms; one of the corresponding integrals is a standard 

form. The residue at z = i is in?/8; I = 1°/8. 
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Applications of complex variables 


In chapter 24, we developed the basic theory of the functions of a complex 
variable, z = x + iy, studied their analyticity (differentiability) properties and 
derived a number of results concerned with values of contour integrals in the 
complex plane. In this current chapter we will show how some of those results 
and properties can be exploited to tackle problems arising directly from physical 
situations or from apparently unrelated parts of mathematics. 

In the former category will be the use of the differential properties of the real 
and imaginary parts of a function of a complex variable to solve problems involv- 
ing Laplace’s equation in two dimensions, whilst an example of the latter might 
be the summation of certain types of infinite series. Other applications, such as 
the Bromwich inversion formula for Laplace transforms, appear as mathematical 
problems that have their origins in physical applications; the Bromwich inversion 
enables us to extract the spatial or temporal response of a system to an initial 
input from the representation of that response in ‘frequency space’ — or, more 
correctly, imaginary frequency space. 

Other topics that will be considered are the location of the (complex) zeros of 
a polynomial, the approximate evaluation of certain types of contour integrals 
using the methods of steepest descent and stationary phase, and the so-called 
‘phase-integral’ solutions to some differential equations. For each of these a brief 
introduction is given at the start of the relevant section and to repeat them here 
would be pointless. We will therefore move on to our first topic of complex 
potentials. 


25.1 Complex potentials 


Towards the end of section 24.2 of the previous chapter it was shown that the real 
and the imaginary parts of an analytic function of z are separately solutions of 
Laplace’s equation in two dimensions. Analytic functions thus offer a possible way 
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Figure 25.1 The equipotentials (dashed circles) and field lines (solid lines) 
for a line charge perpendicular to the z-plane. 


of solving some two-dimensional physical problems describable by a potential 
satisfying V7 = 0. The general method is known as that of complex potentials. 

We also found that if f = u+iv is an analytic function of z then any curve 
u = constant intersects any curve v = constant at right angles. In the context of 
solutions of Laplace’s equation, this result implies that the real and imaginary 
parts of f(z) have an additional connection between them, for if the set of 
contours on which one of them is a constant represents the equipotentials of a 
system then the contours on which the other is constant, being orthogonal to 
each of the first set, must represent the corresponding field lines or stream lines, 
depending on the context. The analytic function f is the complex potential. It 
is conventional to use ¢ and wp (rather than u and v) to denote the real and 
imaginary parts of a complex potential, so that f = d + ip. 

As an example, consider the function 


fi) = —t nz (25.1) 
2m€0 

in connection with the physical situation of a line charge of strength q per unit 

length passing through the origin, perpendicular to the z-plane (figure 25.1). Its 

real and imaginary parts are 


@=—f nz, y= —Zargz. (25.2) 
2n€ 2m€ 


The contours in the z-plane of ¢ = constant are concentric circles and those of 
y = constant are radial lines. As expected these are orthogonal sets, but in addi- 
tion they are, respectively, the equipotentials and electric field lines appropriate to 
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the field produced by the line charge. The minus sign is needed in (25.1) because 
the value of @ must decrease with increasing distance from the origin. 

Suppose we make the choice that the real part @ of the analytic function f 
gives the conventional potential function; y could equally well be selected. Then 
we may consider how the direction and magnitude of the field are related to f. 


> Show that for any complex (electrostatic) potential f(z) the strength of the electric field 
is given by E = |f'(z)| and that its direction makes an angle of x — arg{[f'(z)] with the 
X-axis. 


Because ¢ = constant is an equipotential, the field has components 
E, =-— and Ey =-=. (25.3) 


Since f is analytic, (i) we may use the Cauchy—Riemann relations (24.5) to change the 
second of these, obtaining 


TH) oy. 
Ex, = ee and ear oe (25.4) 
(ii) the direction of differentiation at a point is immaterial and so 
d 6 0 0 
DIS be HY th see (25.5) 


dz Ox Ox Ox 


From these it can be seen that the field at a point is given in magnitude by E = |f’(z)| 
and that it makes an angle with the x-axis given by x — arg[f’(z)]. < 


It will be apparent from the above that much of physical interest can be 
calculated by working directly in terms of f and z. In particular, the electric field 
vector E may be represented, using (25.5) above, by the quantity 


€ =E,+iE, =—[f'(2)’. 


Complex potentials can be used in two-dimensional fluid mechanics problems 
in a similar way. If the flow is stationary (i.e. the velocity of the fluid does not 
depend on time) and irrotational, and the fluid is both incompressible and non- 
viscous, then the velocity of the fluid can be described by V = Vd, where ¢ is the 
velocity potential and satisfies V7 = 0. If, for a complex potential f = ¢ + ip, 
the real part ¢ is taken to represent the velocity potential then the curves p = 
constant will be the streamlines of the flow. In a direct parallel with the electric 
field, the velocity may be represented in terms of the complex potential by 


Va +iV, = Ef, 


the difference of a minus sign reflecting the same difference between the definitions 
of E and V. The speed of the flow is equal to |f’(z)|. Points where f’(z) = 0, and 
thus the velocity is zero, are called stagnation points of the flow. 

Analogously to the electrostatic case, a line source of fluid at z = zo, perpendic- 
ular to the z-plane (i.e. a point from which fluid is emerging at a constant rate), 
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is described by the complex potential 
f(z) =k In(z — zo), 


where k is the strength of the source. A sink is similarly represented, but with k 
replaced by —k. Other simple examples are as follows. 


(i) The flow of a fluid at a constant speed Vo and at an angle « to the x-axis 
is described by f(z) = Vo(exp ia)z. 

(ii) Vortex flow, in which fluid flows azimuthally in an anticlockwise direction 
around some point zo, the speed of the flow being inversely proportional 
to the distance from Zo, is described by f(z) = —ik In(z — Zo), where k is 
the strength of the vortex. For a clockwise vortex k is replaced by —k. 


> Verify that the complex potential 


2 
fe)=%(2+ 5) 


is appropriate to a circular cylinder of radius a placed so that it is perpendicular to a 
uniform fluid flow of speed Vo parallel to the x-axis. 


Firstly, since f(z) is analytic except at z = 0, both its real and imaginary parts satisfy 
Laplace’s equation in the region exterior to the cylinder. Also f(z) > Voz as z > 0, so 
that Re f(z) — Vox, which is appropriate to a uniform flow of speed Vo in the x-direction 
far from the cylinder. 

Writing z = rexpi0 and using de Moivre’s theorem we have 


) sin 0. 


z) are given by 


f(z) =Vo f expi@ + = exp( it) 


@ 
=Vo (++ =) cos8 + iVo (-- 
7 


Thus we see that the streamlines of the flow described by f 


[8 


—_ 


a 
p=Vo (« - “) sin @ = constant. 
r 


In particular, y = 0 on r = a, independently of the value of 0, and so r = a must be a 
streamline. Since there can be no flow of fluid across streamlines, r = a must correspond 
to a boundary along which the fluid flows tangentially. Thus f(z) is a solution of Laplace’s 
equation that satisfies all the physical boundary conditions of the problem, and so, by the 
uniqueness theorem, it is the appropriate complex potential. < 


By a similar argument, the complex potential f(z) = —E(z — a*/z) (note the 
minus signs) is appropriate to a conducting circular cylinder of radius a placed 
perpendicular to a uniform electric field E in the x-direction. 

The real and imaginary parts of a complex potential f = @ + ip have another 
interesting relationship in the context of Laplace’s equation in electrostatics or 
fluid mechanics. Let us choose ¢ as the conventional potential, so that y represents 
the stream function (or electric field, depending on the application), and consider 
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y 


pw | 


Figure 25.2 A curve joining the points P and Q. Also shown is fi, the unit 
vector normal to the curve. 


the difference in the values of y at any two points P and Q connected by some 
path C, as shown in figure 25.2. This difference is given by 


Q 276 F) 
w(Q)—p(P) -| dy -| (+ dx + =~ iy) ’ 
P P OX y 


which, on using the Cauchy—Riemann relations, becomes 


Q A 4 
y(Q) vP)= f ( ao a | ae iy) 


Q 2 2 ad 
= [void = [Pas 


where fi is the vector unit normal to the path C and s is the arc length along the 
path; the last equality is written in terms of the normal derivative 0¢/dn = Vo- ft. 

Now suppose that in an electrostatics application, the path C is the surface of 
a conductor; then 


where o is the surface charge density per unit length normal to the xy-plane. 
Therefore —eo[y(Q) — p(P)] is equal to the charge per unit length normal to the 
xy-plane on the surface of the conductor between the points P and Q. Similarly, 
in fluid mechanics applications, if the density of the fluid is p and its velocity is 
V then 
Q Q 
pln) v(PiI=p | vo-aas=p [’ V-aas 

is equal to the mass flux between P and Q per unit length perpendicular to the 
xy-plane. 
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> A conducting circular cylinder of radius a is placed with its centre line passing through 
the origin and perpendicular to a uniform electric field E in the x-direction. Find the charge 
per unit length induced on the half of the cylinder that lies in the region x < 0. 


As mentioned immediately following the previous example, the appropriate complex 
potential for this problem is f(z) = —E(z — a’/z). Writing z = rexpi0 this becomes 


f(z) =—-E f exp id — & exit] 


2 2 
aG =) cos iE (r+ - ) sin 0, 


so that on r = a the imaginary part of f is given by 


yp = —2Easin 0. 


Therefore the induced charge q per unit length on the left half of the cylinder, between 
0 = 2/2 and 0 = 32/2, is given by 


q = 2e9Ea{[sin(32/2) — sin(z/2)] = —4e9Ea. < 


25.2 Applications of conformal transformations 


In section 24.7 of the previous chapter it was shown that, under a conformal 
transformation w = g(z) from z = x+iy to a new variable w = r+is, if a solution 
of Laplace’s equation in some region R of the xy-plane can be found as the real 
or imaginary part of an analytic function’ of z, then the same expression put in 
terms of r and s will be a solution of Laplace’s equation in the corresponding 
region R’ of the w-plane, and vice versa. In addition, if the solution is constant 
over the boundary C of the region R in the xy-plane, then the solution in the 
w-plane will take the same constant value over the corresponding curve C’ that 
bounds R’. 

Thus, from any two-dimensional solution of Laplace’s equation for a particular 
geometry, typified by those discussed in the previous section, further solutions for 
other geometries can be obtained by making conformal transformations. From 
the physical point of view the given geometry is usually complicated, and so 
the solution is sought by transforming to a simpler one. However, working from 
simpler to more complicated situations can provide useful experience and make 
it more likely that the reverse procedure can be tackled successfully. 


S In fact, the original solution in the xy-plane need not be given explicitly as the real or imaginary 
part of an analytic function. Any solution of V7¢ = 0 in the xy-plane is carried over into another 
solution of V7 = 0 in the new variables by a conformal transformation, and vice versa. 
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(a) z-plane (b) w-plane (c) w-plane 


Figure 25.3 The equipotential lines (broken) and field lines (solid) (a) for an 
infinite charged conducting plane at y = 0, where z = x + iy, and after the 
transformations (b) w = z? and (c) w = z!/? of the situation shown in (a). 


> Find the complex electrostatic potential associated with an infinite charged conducting 
plate y =0, and thus obtain those associated with 
(i) a semi-infinite charged conducting plate (r > 0, s = 0); 


(ii) the inside of a right-angled charged conducting wedge (r > 0, s = O and 
r=0,s>0). 


Figure 25.3(a) shows the equipotentials (broken lines) and field lines (solid lines) for the 
infinite charged conducting plane y = 0. Suppose that we elect to make the real part of 
the complex potential coincide with the conventional electrostatic potential. If the plate is 
charged to a potential V then clearly 


P(x, y) = V —ky, (25.6) 


where k is related to the charge density o by k = a/eo, since physically the electric field E 
has components (0,¢/e9) and E = —V@¢. 

Thus what is needed is an analytic function of z, of which the real part is V —ky. This 
can be obtained by inspection, but we may proceed formally and use the Cauchy—Riemann 
relations to obtain the imaginary part p(x, y) as follows: 


ee! gad! Bee he 


oy Ox Ox oy 
Hence y = kx +c and, absorbing c into V, the required complex potential is 
f(z) =V —ky + ikx = V + ikz. (25.7) 
(i) Now consider the transformation 
w = g(z) =z". (25.8) 


This satisfies the criteria for a conformal mapping (except at z = 0) and carries the upper 
half of the z-plane into the entire w-plane; the equipotential plane y = 0 goes into the 
half-plane r > 0, s = 0. 

By the general results proved, f(z), when expressed in terms of r and s, will give a 
complex potential whose real part will be constant on the half-plane in question; we 
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deduce that 
F(w) = f(z) = V + ikz = V + ikw'? (25.9) 


is the required potential. Expressed in terms of r, s and p = (r? + s?)!/?, w'/? is given by 


1/2 1/2 
12 _ 12) (Ptr .(p-t 5.10 
7 (SF) +() oes 
and, in particular, the electrostatic potential is given by 
k 
(r,s) = Re F(w) =V — 2 [(rP? +2)? =F]. (25.11) 


2 
The corresponding equipotentials and field lines are shown in figure 25.3(b). Using results 
(25.3)-(25.5), the magnitude of the electric field is 
|E| = |F'(w)| = |Sikw7'?| = gk? +°) 4. 
(ii) A transformation ‘converse’ to that used in (1), 
w = g(z) = 24, 


has the effect of mapping the upper half of the z-plane into the first quadrant of the 
w-plane and the conducting plane y = 0 into the wedge r > 0, s =0 andr =0,s>0. 
The complex potential now becomes 


F(w) = V + ikw? 


= V +ik[(r? — s) + 2irs], (25.12) 
showing that the electrostatic potential is V—2krs and that the electric field has components 
E = (2ks, 2kr). (25.13) 


Figure 25.3(c) indicates the approximate equipotentials and field lines. (Note that, in both 
transformations, g/(z) is either 0 or co at the origin, and so neither transformation is 
conformal there. Consequently there is no violation of result (ii), given at the start of 
section 24.7, concerning the angles between intersecting lines.) < 


The method of images, discussed in section 21.5, can be used in conjunction with 
conformal transformations to solve some problems involving Laplace’s equation 
in two dimensions. 


>A wedge of angle n/a with its vertex at z = 0 is formed by two semi-infinite conducting 
plates, as shown in figure 25.4(a). A line charge of strength q per unit length is positioned 


at z = Zo, perpendicular to the z-plane. By considering the transformation w = z*, find the 
complex electrostatic potential for this situation. 


Let us consider the action of the transformation w = z* on the lines defining the positions 
of the conducting plates. The plate that lies along the positive x-axis is mapped onto the 
positive r-axis in the w-plane, whereas the plate that lies along the direction exp(iz/a) is 
mapped into the negative r-axis, as shown in figure 25.4(b). Similarly the line charge at zo 
is mapped onto the point wo = 2). 

From figure 25.4(b), we see that in the w-plane the problem can be solved by introducing 
a second line charge of opposite sign at the point wo, so that the potential ® = 0 along 
the r-axis. The complex potential for such an arrangement is simply 


F(w) = sages In(w — wo) + ss 
2n€o 2n€0 


In(w — wo). 
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(a) (b) 4 Wo 


Figure 25.4 (a) An infinite conducting wedge with interior angle z/a« and a 
line charge at z = Zo; (b) after the transformation w = z%, with an additional 
image charge placed at w = wo. 


Substituting w = z*% into the above shows that the required complex potential in the 


original z-plane is 
q Zt — 20" 
= 1 i 
f@) 2n€o = ( ZEAE ) 


It should be noted that the appearance of a complex conjugate in the final 
expression is not in conflict with the general requirement that the complex 
potential be analytic. It is z” that must not appear; here, z)* is no more than a 
parameter of the problem. 


25.3 Location of zeros 


The residue theorem, relating the value of a closed contour integral to the sum of 
the residues at the poles enclosed by the contour, was discussed in the previous 
chapter. One important practical use of an extension to the theorem is that of 
locating the zeros of functions of a complex variable. The location of such zeros 
has a particular application in electrical network and general oscillation theory, 
since the complex zeros of certain functions (usually polynomials) give the system 
parameters (usually frequencies) at which system instabilities occur. As the basis 
of a method for locating these zeros we next prove three important theorems. 

(i) If f(z) has poles as its only singularities inside a closed contour C and is 
not zero at any point on C then 


, a dz = 2D a (25.14) 


Here Nj is the order of the jth zero of f(z) enclosed by C. Similarly P; is the 
order of the jth pole of f(z) inside C. 
To prove this we note that, at each position z;, f(z) can be written as 


f(z) =(2—2;)" (2), (25.15) 
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where #(z) is analytic and non-zero at z = z; and mj is positive for a zero and 
negative for a pole. Then the integrand f’(z)/f(z) takes the form 
7 / 

PY ames 4 OC). (25.16) 

f(z) z—-2j blz) 
Since $(z;) # 0, the second term on the RHS is analytic; thus the integrand has a 
simple pole at z = z;, with residue m;. For zeros mj = N; and for poles mj; = —P), 
and thus (25.14) follows from the residue theorem. 

(ii) If f(z) is analytic inside C and not zero at any point on it then 


2n S> Nj = Aclarg f(2)), (25.17) 
Jj 


where Ac[x] denotes the variation in x around the contour C. 
Since f is analytic, there are no P;; further, since 


f@_d 
Aa q, Ens). (25.18) 
equation (25.14) can be written 
: f'(z) . 
2ni "Ny = FG) dz = Ac([Lnf(z)]. (25.19) 
However, 
Ac[Ln f(z)] = Ac[In |f(z)I] + iAclarg f(2)], (25.20) 


and, since C is a closed contour, In|f(z)| must return to its original value; so the 
real term on the RHS is zero. Comparison of (25.19) and (25.20) then establishes 
(25.17), which is known as the principle of the argument. 

(iii) If f(z) and g(z) are analytic within and on a closed contour C and 
|g(z)| < |f(z)| on C then f(z) and f(z) + g(z) have the same number of zeros 
inside C; this is Rouché’s theorem. 

With the conditions given, neither f(z) nor f(z) + g(z) can have a zero on C. 
So, applying theorem (ii) with an obvious notation, 


andi Nf +8) = Aclarg(f + g)] 
= Acfarg f] + Ac[arg(1 + g/f)] 
= Indo, Ne(f) + Aclarg(l + g/f)]. (25.21) 


Further, since |g| < |f| on C, 1+ g/f always lies within a unit circle centred 
on z = 1; thus its argument always lies in the range —7/2 < arg(1+¢/f) < 2/2 
and cannot change by any multiple of 2z. It must therefore return to its 
original value when z returns to its starting point having traversed C. Hence 
the second term on the RHS of (25.21) is zero and the theorem is estab- 
lished. 

The importance of Roucheé’s theorem is that for some functions, in particular 
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polynomials, only the behaviour of a single term in the function need be con- 
sidered if the contour is chosen appropriately. For example, for a polynomial, 
f(z)+2(z) = a biz', only the properties of its largest power, taken as f(z), need 
be investigated if a circular contour is chosen with radius R sufficiently large that, 
on the contour, the magnitude of the largest power term, |by RI, is greater than 
the sum of the magnitudes of all other terms. It is obvious that f(z) = byz™ has 
N zeros inside |z| = R (all at the origin); consequently, f + g also has N zeros 
inside the same circle. 

The corresponding situation, in which only the properties of the polynomial’s 
smallest power, again taken as f(z), need be investigated is a circular contour 
with a radius R chosen sufficiently small that, on the contour, the magnitude of 
the smallest power term (usually the constant term in a polynomial) is greater 
than the sum of the magnitudes of all other terms. Then, a similar argument to 
that given above shows that, since f(z) = bo has no zeros inside |z| = R, neither 
does f +g. 

A weak form of the maximum-modulus theorem may also be deduced. This 
states that if f(z) is analytic within and on a simple closed contour C then |f(z)| 
attains its maximum value on the boundary of C. The proof is as follows. 

Let |f(z)| < M on C with equality at at least one point of C. Now suppose 
that there is a point z = a inside C such that |f(a)| > M. Then the function 
h(z) = f(a) is such that |h(z)| > | — f(z)| on C, and thus, by Rouché’s theorem, 
h(z) and h(z) — f(z) have the same number of zeros inside C. But h(z) (= f(a)) 
has no zeros inside C, and, again by Rouché’s theorem, this would imply that 
f(a) — f(z) has no zeros in C. However, f(a) — f(z) clearly has a zero at z =a, 
and so we have a contradiction; the assumption of the existence of a point z =a 
inside C such that |f(a)| > M must be invalid. This establishes the theorem. 

The stronger form of the maximum-modulus theorem, which we do not prove, 
states, in addition, that the maximum value of f(z) is not attained at any interior 
point except for the case where f(z) is a constant. 


> Show that the four zeros of h(z) = z*-+2z+1 occur one in each quadrant of the Argand 
diagram and that all four lie between the circles \z\| = 2/3 and |z| = 3/2. 


Putting z = x and z = iy shows that no zeros occur on the real or imaginary axes. They 
must therefore occur in conjugate pairs, as can be shown by taking the complex conjugate 
of h(z) = 0. 

Now take C as the contour OXYO shown in figure 25.5 and consider the changes 
A[argh] in the argument of h(z) as z traverses C. 


(i) OX: argh is everywhere zero, since h is real, and thus Aox[argh] = 0. 
(ii) XY: z = Rexpi0 and so argh changes by an amount 


Axy [argh] = Ayy [argz*] + Ayy [arg(1 + 2° +2) 
= Axy [arg R’e*”] + Ayy {arg[1 + O(R*)]} 
=2n+O(R™). (25.22) 
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O 


Figure 25.5 A contour for locating the zeros of a polynomial that occur in 
the first quadrant of the Argand diagram. 


(iii) YO: z = iy and so argh = tan"! y/(y* + 1), which starts at O(R~) and finishes at 
0 as y goes from large R to 0. It never reaches z/2 because y* + 1 = 0 has no real 
positive root. Thus Ayo [argh] = 0. 


Hence for the complete contour Ac [argh] = 0+ 2x +0+ O(R~), and, if R is allowed 
to tend to infinity, we deduce from (25.17) that h(z) has one zero in the first quadrant. 
Furthermore, since the roots occur in conjugate pairs, a second root must lie in the fourth 
quadrant, and the other pair must lie in the second and third quadrants. 

To show that the zeros lie within the given annulus in the z-plane we apply Rouché’s 
theorem, as follows. 


(i) With C as |z| = 3/2, f = 24, g =z4+1. Now |f| = 81/16 on C and |g| < 1+ |z| < 
5/2 < 81/16. Thus, since z* = 0 has four roots inside |z| = 3/2, so also does 
zt+z+1=0. 

(ii) With C as |z| = 2/3, f = 1, g =z*++z. Now f =1 on C and |g| < |z4/ +|z| = 
16/81 + 2/3 = 70/81 < 1. Thus, since f = 0 has no roots inside |z| = 2/3, neither 
does 1+2z+2+=0. 


Hence the four zeros of h(z) = z4 + 2+ 1 occur one in each quadrant and all lie between 
the circles |z| = 2/3 and |z| = 3/2. < 


A further technique useful for locating the zeros of functions is explained in 
exercise 25.8. 


25.4 Summation of series 


We now turn to an application of contour integration which at first sight might 
seem to lie in an unrelated area of mathematics, namely the summation of infinite 
series. Sometimes a real infinite series with index n, say, can be summed with the 
help of a suitable complex function that has poles on the real axis at the various 
positions z = n with the corresponding residues at those poles equal to the values 
of the terms of the series. A worked example provides the best explanation of 
how the technique is applied; other examples will be found in the exercises. 
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> By considering 


i Ae 
@ (@teap 


where a is not an integer and C is a circle of large radius, evaluate 


= 1 
1B (a+n)" 


n=—0o 


The integrand has (i) simple poles at z = integer n, for —oo < n < «, due to the factor 
cot zz and (ii) a double pole at z = —a. 
(i) To find the residue of cot zz, put z =n+ é for small é: 
cos(nz + €2) cos nt 1 


cotrz2 = —————— 2 FE 
sin(na + €7) (cosna)én én 


The residue of the integrand at z =n is thus x(a+n)-?x7!. 
(ii) Putting z = —a + € for small € and determining the coefficient of €~! gives’ 


ncot nz Tt 
G@aae =a cot(—az + Ez) 


d 
=5 cot(—az) + € | —(cot1z) ee ae 
i dz ny 
so that the residue at the double pole z = —a is given by 
n[—n cosec *72z]--, = —1 cosec 77a. 


Collecting together these results to express the residue theorem gives 


N 


l= ¢ Oe dz = 2ni », = 1’ cosec ne : (25.23) 


c (a+zy (a+n)? 


=—N 


ne 


where N equals the integer part of R. But as the radius R of C tends to «, cotaz > Fi 
(depending on whether Im z is greater or less than zero, respectively). Thus 


dz 
I <k f 


which tends to 0 as R > oo. Thus I > 0 as R (and hence N) — o, and (25.23) establishes 
the result 


oe) 2 


1 T 
2 ra 


Rar sin? za 
Series with alternating signs in the terms, ie. (—1)", can also be attempted 


in this way but using cosec mz rather than cot zz, since the former has residue 
(—1)"x7~! at z =n (see exercise 25.11). 


S$ This again illustrates one of the techniques for determining residues. 
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Ims 


Figure 25.6 The integration path of the inverse Laplace transform is along 
the infinite line L. The quantity 2 must be positive and large enough for all 
poles of the integrand to lie to the left of L. 


25.5 Inverse Laplace transform 


As a further example of the use of contour integration we now discuss a method 
whereby the process of Laplace transformation, discussed in chapter 13, can be 
inverted. 

It will be recalled that the Laplace transform f(s) of a function f(x), x > 0, is 
given by 


f(s)= fe e *f(x) dx, Re s > 50. (25.24) 


In chapter 13, functions f(x) were deduced from the transforms by means of a 
prepared dictionary. However, an explicit formula for an unknown inverse may 
be written in the form of an integral. It is known as the Bromwich integral and is 
given by 


fx) = as i cas e*F(s) ds,  A>0, (25.25) 


2ni A—ico 


where s is treated as a complex variable and the integration is along the line L 
indicated in figure 25.6. The position of the line is dictated by the requirements 
that / is positive and that all singularities of f(s) lie to the left of the line. 

That (25.25) really is the unique inverse of (25.24) is difficult to show for general 
functions and transforms, but the following verification should at least make it 
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e le 
Lay aes 
; h 
L . 56 s 
Loomer e Lee eat 


(a) (b) (c) 


Figure 25.7 Some contour completions for the integration path L of the 
inverse Laplace transform. For details of when each is appropriate see the 
main text. 


plausible: 


A+io ioe) 
‘jas / dse™ e™f(u) du, - Re(s) > 0, ie. A> 0, 
2ni 0 


A-i” 
1 00 A+in 
= =| du f(u) e—) ds 
2ni Jo dio 
1 % Oh. . 
= =| du fw) | eX) POY; dy, putting s = 2+ ip, 
2ni 0 —oco 


1 of? ; 
eo Ae A(x—u) = 
x= | fiwe 2n6(x — u) du 


= ey (25.26) 
0 x<0. 

Our main purpose here is to demonstrate the use of contour integration. To 

employ it in the evaluation of the line integral (25.25), the path L must be made 

part of a closed contour in such a way that the contribution from the completion 

either vanishes or is simply calculable. 

A typical completion is shown in figure 25.7(a) and would be appropriate if 
f(s) had a finite number of poles. For more complicated cases, in which f(s) has 
an infinite sequence of poles but all to the left of L as in figure 25.7(b), a sequence 
of circular-arc completions that pass between the poles must be used and f(x) is 
obtained as a series. If f(s) is a multivalued function then a cut plane is needed 
and a contour such as that shown in figure 25.7(c) might be appropriate. 

We consider here only the simple case in which the contour in figure 25.7(a) 
is used; we refer the reader to the exercises at the end of the chapter for others. 
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Ideally, we would like the contribution to the integral from the circular arc T' to 
tend to zero as its radius R > oo. Using a modified version of Jordan’s lemma, 
it may be shown that this is indeed the case if there exist constants M > 0 and 
a >0 such that on 


FOI < Fe 


Moreover, this condition always holds when f(s) has the form 


where P(s) and Q(s) are polynomials and the degree of Q(s) is greater than that 
of P(s). 

When the contribution from the part-circle [ tends to zero as R — o, we 
have from the residue theorem that the inverse Laplace transform (25.25) is given 
simply by 


f= Py (residues of f(s)e* at all poles) . (25.27) 


> Find the function f(x) whose Laplace transform is 


i()=s—> 


where k is a constant. 


It is clear that f(s) is of the form required for the integral over the circular arc I to tend 
to zero as R > ~, and so we may use the result (25.27) directly. Now 
se 


I)" = We +h’ 


and thus has simple poles at s = k and s = —k. Using (24.57) the residues at each pole can 
be easily calculated as 


SX 


kek ke“ 
Tk and R(—k) = Tk 


Thus the inverse Laplace transform is given by 
f(x) = 4 (e& +e™) = coshkx. 


This result may be checked by computing the forward transform of coshkx. < 


Rk) = 


Sometimes a little more care is required when deciding in which half-plane to 
close the contour C. 


> Find the function f(x) whose Laplace transform is 


I(s) = (e® —e), 


where a and b are fixed and positive, with b > a. 


From (25.25) we have the integral 


1 A+i00 eas =_ elx—b)s 
f@) = 55 | ds. (25.28) 


Ji 
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F(x) 


a b x 


Figure 25.8 The result of the Laplace inversion of f(s) = s“!(e~® —e~) with 
b>a. 


Now, despite appearances to the contrary, the integrand has no poles, as may be confirmed 
by expanding the exponentials as Taylor series about s = 0. Depending on the value of x, 
several cases arise. 

(i) For x < a both exponentials in the integrand will tend to zero as Re s > oo. Thus 
we may close L with a circular arc [ in the right half-plane (A can be as small as desired), 
and we observe that s x integrand tends to zero everywhere on I as R > o. With no 
poles enclosed and no contribution from I, the integral along L must also be zero. Thus 


f(x) =0 for x <a. (25.29) 


(ii) For x > b the exponentials in the integrand will tend to zero as Re s > —oo, and 
so we may close L in the left half-plane, as in figure 25.7(a). Again the integral around 
vanishes for infinite R, and so, by the residue theorem, 


f(x) =0 for x > b. (25.30) 


(iii) For a < x < b the two parts of the integrand behave in different ways and have to 
be treated separately: 


I b= 1 elx—a)s F 1 elx—b)s 
nn) 1. nr i) 7 i. (ar 


s-— 


The integrand of I, then vanishes in the far left-hand half-plane, but does now have a 
(simple) pole at s = 0. Closing L in the left half-plane, and using the residue theorem, we 
obtain 


I, = residue at s=0 of s'e0-@ = 1, (25.31) 
The integrand of I, however, vanishes in the far right-hand half-plane (and also has 
a simple pole at s = 0) and is evaluated by a circular-arc completion in that half-plane. 


Such a contour encloses no poles and leads to I, = 0. 
Thus, collecting together results (25.29)-(25.31) we obtain 


0 forx <a, 
f(x)=4 1 fora<x<b, 
0 forx>b, 


as shown in figure 25.8. < 


887 


APPLICATIONS OF COMPLEX VARIABLES 


25.6 Stokes’ equation and Airy integrals 


Much of the analysis of situations occurring in physics and engineering is con- 
cerned with what happens at a boundary within or surrounding a physical system. 
Sometimes the existence of a boundary imposes conditions on the behaviour of 
variables describing the state of the system; obvious examples include the zero 
displacement at its end-points of an anchored vibrating string and the zero 
potential contour that must coincide with a grounded electrical conductor. 

More subtle are the effects at internal boundaries, where the same non-vanishing 
variable has to describe the situation on either side of the boundary but its 
behaviour is quantitatively, or even qualitatively, different in the two regions. In 
this section we will study an equation, Stokes’ equation, whose solutions have 
this latter property; as well as solutions written as series in the usual way, we will 
find others expressed as complex integrals. 

The Stokes’ equation can be written in several forms, e.g. 


+Axy =0; 
We will adopt the last of these, but write it as 


2 

a =zy (25.32) 
to emphasis that its complex solutions are valid for a complex independent 
variable z, though this also means that particular care has to be exercised when 
examining their behaviour in different parts of the complex z-plane. The other 
forms of Stokes’ equation can all be reduced to that of (25.32) by suitable 
(complex) scaling of the independent variable. 


25.6.1 The solutions of Stokes’ equation 


It will be immediately apparent that, even for z restricted to be real and denoted 
by x, the behaviour of the solutions to (25.32) will change markedly as x passes 
through x = 0. For positive x they will have similar characteristics to the solutions 
of y"” = k?y, where k is real; these have monotonic exponential forms, either 
increasing or decreasing. On the other hand, when x is negative the solutions 
will be similar to those of y” + k?y = 0, ie. oscillatory functions of x. This is 
just the sort of behaviour shown by the wavefunction describing light diffracted 
by a sharp edge or by the quantum wavefunction describing a particle near to 
the boundary of a region which it is classically forbidden to enter on energy 
grounds. Other examples could be taken from the propagation of electromagnetic 
radiation in an ion plasma or wave-guide. 

Let us examine in a bit more detail the behaviour of plots of possible solutions 
y(z) of Stokes’ equation in the region near z = 0 and, in particular, what may 
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Figure 25.9 Behaviour of the solutions y(z) of Stokes’ equation near z = 0 
for various values of 24 = —y’(0). (a) with 2 small, (b) with / large and (c) with 
4 appropriate to the Airy function Ai(z). 


happen in the region z > 0. For definiteness and ease of illustration (see figure 
25.9), let us suppose that both y and z, and hence the derivatives of y, are real and 
that y(0) is positive; if it were negative, our conclusions would not be changed 
since equation (25.32) is invariant under y(z) > —y(z). The only difference would 
be that all plots of y(z) would be reflected in the z-axis. 

We first note that d*y/dx*, and hence also the curvature of the plot, has the 
same sign as Z, ie. it has positive curvature when z > 0, for so long as y(z) 
remains positive there. What will happen to the plot for z > 0 therefore depends 
crucially on the value of y’/(0). If this slope is positive or only slightly negative 
the positive curvature will carry the plot, either immediately or ultimately, further 
away from the z-axis. On the other hand, if y’(0) is negative but sufficiently large 
in magnitude, the plot will cross the y = 0 line; if this happens the sign of the 
curvature reverses and again the plot will be carried ever further from the z-axis, 
only this time towards large negative values. 

Between these two extremes it seems at least plausible that there is a particular 
negative value of y’(0) that leads to a plot that approaches the z-axis asymptot- 
ically, never crosses it (and so always has positive curvature), and has a slope 
that, whilst always negative, tends to zero in magnitude. There is such a solu- 
tion, known as Ai(z), whose properties we will examine further in the following 
subsections. The three cases are illustrated in figure 25.9. 

The behaviour of the solutions of (25.32) in the region z < 0 is more straight- 
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forward, in that, whatever the sign of y at any particular point z, the curvature 
always has the opposite sign. Consequently the curve always bends towards the 
z-axis, crosses it, and then bends towards the axis again. Thus the curve exhibits 
oscillatory behaviour. Furthermore, as —z increases, the curvature for any given 
|y| gets larger; as a consequence, the oscillations become increasingly more rapid 
and their amplitude decreases. 


25.6.2 Series solution of Stokes’ equation 


Obtaining a series solution of Stokes’ equation presents no particular difficulty 
when the methods of chapter 16 are used. The equation, written in the form 


has no singular points except at z = oo. Every other point in the z-plane is 
an ordinary point and so two linearly independent series expansions about it 
(formally with indicial values o = 0 and o = 1) can be found. Those about z = 0 
take the forms )7f a,z” and 7p b,z"*!, The corresponding recurrence relations 
are 


(n za 3)(n a 2)an+3 =a, and (n+4)(n + 3)bn43 = by, 


and the two series (with ay = by = 1) take the forms 


3 6 


yi) =l+ ot te, 
(3)(2) — (6)(5)(3)(2) 
zt z! 
BA=2+ Oat HOE * 
The ratios of successive terms for the two series are thus 
Ay 43.z"*3 73 by43z"t4 73 


ag @Laae2 * baz! (nt 4n +3) 
It follows from the ratio test that both series are absolutely convergent for all z. 
A similar argument shows that the series for their derivatives are also absolutely 
convergent for all z. Any solution of the Stokes’ equation is representable as a 
superposition of the two series and so is analytic for all finite z; it is therefore an 
integral function with its only singularity at infinity. 


25.6.3 Contour integral solutions 


We now move on to another form of solution of the Stokes’ equation (25.32), one 
that takes the form of a contour integral in which z appears as a parameter in 
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the integrand. Consider the contour integral 


b 
y(z) = / f(t) exp(zt) dt, (25.33) 


in which a, b and f(t) are all yet to be chosen. Note that the contour is in the 
complex t-plane and that the path from a to b can be distorted as required so 
long as no poles of the integrand are trapped between an original path and its 
distortion. 

Substitution of (25.33) into (25.32) yields 


b b 
/ t° f(t) exp(zt)dt = . z f(t) exp(zt) dt 


b 1 7 
= [ f(t) exp(zt)]? — / _ sepeiak 


If we could choose the limits a and b so that the end-point contributions vanish 
then Stokes’ equation would be satisfied by (25.33), provided f(t) satisfies 


aso 


+P f(t) = f(t) = Aexp(—483), (25.34) 


where A is any constant. 

To make the end-point contributions vanish we must choose a and b such that 
exp(—4$0° + zt) = 0 for both values of t. This can only happen if |a| — oo and 
|b| oo and, even then, only if Re (t?) is positive. This condition is satisfied if 


2nn — 51 < 3arg(t) < 2na + 30 for some integer n. 


Thus a and b must each be at infinity in one of the three shaded areas shown in 
figure 25.10, but clearly not in the same area as this would lead to a zero value 
for the contour integral. This leaves three contours (marked C;, Cy and C3 in the 
figure) that start and end in different sectors. However, only two of them give rise 
to independent integrals since the path C2 + C3 is equivalent to (can be distorted 
into) the path C;. 

The two integral functions given particular names are 


1 
Ai(z) = — | exp(—4t? + zt) dt (25.35) 
2ni Jc, 
and 


Bi(z) = = | exp(—4° + zt) dt — al exp(—40° + zt) dt. 
qi 3 (25.36) 


Stokes’ equation is unchanged if the independent variable is changed from z 
to €, where € = exp(2zi/3)z = Qz. This is also true for the repeated change 
z — QC = Q?z. The same changes of variable, rotations of the complex plane 
through 27/3 or 42/3, carry the three contours C;, C) and C3 into each other, 
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Im t 


Figure 25.10 The contours used in the complex t-plane to define the functions 
Ai(z) and Bi(z). 


though sometimes the sense of traversal is reversed. Consequently there are 
relationships connecting Ai and Bi when the rotated variables are used as their 
arguments. As two examples, 


Ai(z) + QAi(Qz) + Q’Ai(Q?z) = 0, (25.37) 


Bi(z) = i[ Q’ Ai(Q*z) — NAi(Qz)] = e™/PAi(ze 7") + e*/Ai(ze™?)., 
(25.38) 


Since the only requirements for the integral paths is that they start and end in 
the correct sectors, we can distort path C; so that it lies on the imaginary axis 
for virtually its whole length and just to the left of the axis at its two ends. This 
enables us to obtain an alternative expression for Ai(z), as follows. 

Setting t = is, where s is real and —oo < s < «©, converts the integral represen- 
tation of Ai(z) to 


Ai(z) = x [ exp[i(4s* + zs)]ds. 


2 
Now, the exponent in this integral is an odd function of s and so the imaginary 
part of the integrand contributes nothing to the integral. What is left is therefore 


MoS [ “cos(4s? + 2s) ds. (25.39) 


This form shows explicitly that when z is real, so is Ai(z). 
This same representation can also be used to justify the association of the 
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contour integral (25.35) with the particular solution of Stokes’ equation that 
decays monotonically to zero for real z > 0 as |z| — 00. As discussed in subsection 
25.6.1, all solutions except the one called Ai(z) tend to -too as z (real) takes on 
increasingly large positive values and so their asymptotic forms reflect this. In a 
worked example in subsection 25.8.2 we use the method of steepest descents (a 
saddle-point method) to show that the function defined by (25.39) has exactly 
the characteristic asymptotic property expected of Ai(z) (see page 911). It follows 
that it is the same function as Ai(z), up to a real multiplicative constant. 

The choice of definition (25.36) as the other named solution Bi(z) of Stokes’ 
equation is a less obvious one. However, it is made on the basis of its behaviour for 
negative real values of z. As discussed earlier, Ai(z) oscillates almost sinusoidally 
in this region, except for a relatively slow increase in frequency and an even 
slower decrease in amplitude as —z increases. The solution Bi(z) is chosen to be 
the particular function that exhibits the same behaviour as Ai(z) except that it is 
in quadrature with Ai, ie. it is x/2 out of phase with it. Specifically, as x > —oo, 


: 1 . (2x3? 2 

Ai(x) ~ ——— ee 25.40 
i(x) ani sin ( 3 + a)> ( ) 
E 1 2\xP/2 ox 

Bi(x) ~ A cos ( 3 + a): (25.41) 


There is a close parallel between this choice and that of taking sine and cosine 
functions as the basic independent solutions of the simple harmonic oscillator 
equation. Plots of Ai(z) and Bi(z) for real z are shown in figure 25.11. 


> By choosing a suitable contour for C, in (25.35), express Ai(O) in terms of the gamma 
function. 


With z set equal to zero, (25.35) takes the form 
Ai(0) = = | exp(—413) dt. 
2ni Jc, ? 


We again use the freedom to choose the specific line of the contour so as to make the 
actual integration as simple as possible. 

Here we consider C; as made up of two straight-line segments: one along the line 
argt = 47/3, starting at infinity in the correct sector and ending at the origin; the other 
starting at the origin and going to infinity along the line argt = 27/3, thus ending in the 
correct final sector. On each, we set x0 = s, where s is real and positive on both lines. 
Then dt = e47'/3(3s)-?? ds on the first segment and dt = e?*/3(3s)-2/3 ds on the second. 
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Tre 


"ha, 
ee, 


The functions Ai(z) (full line) and Bi(z) (broken line) for real 


Figure 25.11 
Z. 


Then we have 


1 f° 1 f® 
Ai(0) = — / eo Se8*3(35)73 ds + — i e8e*'3(35)7 ds 
2ni Jy 2ni Jo 
_ a We o8(— e834 @8!/3) 5-273 ds 
0 
Pf capita 
2ni 0 
3-1/6 i 
= T(3), 


2n 


where we have used the standard integral defining the gamma function in the last line. < 


Finally in this subsection we should mention that the Airy functions and their 
derivatives are closely related to Bessel functions of orders +4 and +4 and that 
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there exist many representations, both as linear combinations and as indefinite 
integrals, of one in terms of the other 


25.7 WKB methods 


Throughout this book we have had many occasions on which it has been necessary 
to solve the equation 
2 

ad +kif(x)y =0 (25.42) 

dx 
when the notionally general function f(x) has been, in fact, a constant, usually 
the unit function f(x) = 1. Then the solutions have been elementary and of the 
form Asinkox or Acoskox with arbitrary but constant amplitude A. 

Explicit solutions of (25.42) for a non-constant f(x) are only possible in a 
limited number of cases, but, as we will show, some progress can be made if f(x) 
is a slowly varying function of x, in the sense that it does not change much in a 
range of x of the order of ko!. 

We will also see that it is possible to handle situations in which f(x) is 
complex; this enables us to deal with, for example, the passage of waves through 
an absorbing medium. Developing such solutions will involve us in finding the 
integrals of some complex quantities, integrals that will behave differently in the 
various parts of the complex plane — hence their inclusion in this chapter. 


25.7.1 Phase memory 


Before moving on to the formal development of WKB methods! we discuss the 
concept of phase memory which is the underlying idea behind them. 

Let us first suppose that f(x) is real, positive and essentially constant over 
a range of x and define n(x) as the positive square root of f(x); n(x) is then 
also real, positive and essentially constant over the same range of x. We adopt 
this notation so that the connection can be made with the description of an 
electromagnetic wave travelling through a medium of dielectric constant f(x) 
and, consequently, refractive index n(x). The quantity y(x) would be the electric 
or magnetic field of the wave. For this simplified case, in which we can omit the 


S These relationships and many other properties of the Airy functions can be found in, for example, 
M. Abramowitz and I. A. Stegun (eds), Handbook of Mathematical Functions (New York: Dover, 
1965) pp. 446-50. 


1 So called because they were used, independently, by Wentzel, Kramers and Brillouin to tackle 
certain wave-mechanical problems in 1926, though they had earlier been studied in some depth by 
Jeffreys and used as far back as the first half of the nineteenth century by Green. 
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x-dependence of n(x), the solution would be (as usual) 
y(x) = A exp(—ikonx), (25.43) 


with both A and n constant. The quantity kgnx would be real and would be called 
the ‘phase’ of the wave; it increases linearly with x. 

As a first variation on this simple picture, we may allow f(x) to be complex, 
though, for the moment, still constant. Then n(x) is still a constant, albeit a 
complex one: 


n=ut+i. 
The solution is formally the same as before; however, whilst it still exhibits 


oscillatory behaviour, the amplitude of the oscillations either grows or declines, 
depending upon the sign of v: 


y(x) = A exp(—ikonx) = A exp[—iko(u + iv)x] = Aexp(kovx) exp(—ikopx). 


This solution with v negative is the appropriate description for a wave travelling 
in a uniform absorbing medium. The quantity ko(u + iv)x is usually called the 
complex phase of the wave. 

We now allow f(x), and hence n(x), to be both complex and varying with 
position, though, as we have noted earlier, there will be restrictions on how 
rapidly f(x) may vary if valid solutions are to be obtained. The obvious extension 
of solution (25.43) to the present case would be 


y(x) = A exp[—ikon(x)x], (25.44) 


but direct substitution of this into equation (25.42) gives 


y" +key = k(n'?x? + 2nn'x)y — iko(n"x + 2n’)y. 


Clearly the RHS can only be zero, as is required by the equation, if n’ and n” are 
both very small, or if some unlikely relationship exists between them. 

To try to improve on this situation, we consider how the phase ¢ of the solution 
changes as the wave passes through an infinitesimal thickness dx of the medium. 
The infinitesimal (complex) phase change d@ for this is clearly kon(x)dx, and 
therefore will be 


Ad =ko [ n(u) du 


for a finite thickness x of the medium. This suggests that an improvement on 
(25.44) might be 


y(x) = Aexp (—ito f nu) di) : (25.45) 
0 
This is still not an exact solution as now 
y" (x) + kon?(x)y(x) = —ikon! (x) v(x). 
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This still requires kon'(x) to be small (compared with, say, kan?(x)), but is some 
improvement (not least in complexity!) on (25.44) and gives some measure of the 
conditions under which the solution might be a suitable approximation. 

The integral in equation (25.45) embodies what is sometimes referred to as the 
phase memory approach; it expresses the notion that the phase of the wave-like 
solution is the cumulative effect of changes it undergoes as it passes through the 
medium. If the medium were uniform the overall change would be proportional 
to nx, as in (25.43); the extent to which it is not uniform is reflected in the amount 
by which the integral differs from nx. 

The condition for solution (25.45) to be a reasonable approximation can be 
written as n’ky! <n? or, in words, the change in n over an x-range of kj! should 
be small compared with n’. For light in an optical medium, this means that the 
refractive index n, which is of the order of unity, must change very little over a 
distance of a few wavelengths. 

For some purposes the above approximation is adequate, but for others further 
refinement is needed. This comes from considering solutions that are still wave- 
like but have amplitudes, as well as phases, that vary with position. These are the 
WKB solutions developed and studied in the next three subsections. 


25.7.2. Constructing the WKB solutions 


Having formulated the notion of phase memory, we now construct the WKB 
solutions of the general equation (25.42), in which f(x) can now be both position- 
dependent and complex. As we have already seen, it is the possibility of a complex 
phase that permits the existence of wave-like solutions with varying amplitudes. 
Since n(x) is calculated as the square root of f(x), there is an ambiguity in its 
overall sign. In physical applications this is normally resolved unambiguously by 
considerations such as the inevitable increase in entropy of the system, but, so far 
as dealing with purely mathematical questions is concerned, the ambiguity must 
be borne in mind. 

The process we adopt is an iterative one based on the assumption that the 
second derivative of the complex phase with respect to x is very small and can 
be approximated at each stage of the iteration. So we start with equation (25.42) 
and look for a solution of the form 


V(x) = AexpliP(x)], (25.46) 

where A is a constant. When this is substituted into (25.42) the equation becomes 
do\* br _ 

(=) +iG + kgn“(x) | y(x) = 0. (25.47) 


Setting the quantity in square brackets to zero produces a non-linear equation for 
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which there is no obvious solution for a general n(x). However, on the assumption 
that d°p/dx? is small, an iterative solution can be found. 
As a first approximation ¢” is ignored, and the solution 


ae + kon(x) 
dx 


is obtained. From this, differentiation gives an approximate value for 


vo 7 dn 

dee dx’ 
which can be substituted into equation (25.47) to give, as a second approximation 
for dd/dx, the expression 


1/2 
ae zt ac an ka 


dx dx 
= thn (14a t+) 
x= +kon+ ~ a 
This can now be integrated to give an approximate expression for (x) as follows: 
P(x) = Xko | n(u) du + ; In[ n(x) ], (25.48) 


where the constant of integration has been formally incorporated into the lower 
limit x9 of the integral. Now, noting that exp(i $iln n) = n—'/?, substitution of 
(25.48) into equation (25.46) gives 


A ; 
y(x) = a2 exp | +o ff 


Xo 


x 


n(u) in| (25.49) 


as two independent WKB solutions of the original equation (25.42). This result is 
essentially the same as that in (25.45) except that the amplitude has been divided 
by /n(x), ie. by [f(x)]!/4. Since f(x) may be complex, this may introduce an 
additional x-dependent phase into the solution as well as the more obvious change 
in amplitude. 


> Find two independent WKB solutions of Stokes’ equation in the form 


aa 


re + Axy =0,with 1 real and > 0. 


The form of the equation is the same as that in (25.42) with f(x) = x, and therefore 
n(x) = x!/?. The WKB solutions can be read off immediately using (25.49), so long as 
we remember that although f(x) is real, it has four fourth roots and that therefore the 
constant appearing in a solution can be complex. Two independent WKB solutions are 


Ax Morin ie Ax _ 24 3/2 
aT exp [ivi [ vids | mira exp | ; 33/2 | 


|x| 


ya (x) = 
(25.50) 
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The precise combination of these two solutions that is required for any particular problem 
has to be determined from the problem. < 


When Stokes’ equation is applied more generally to functions of a complex 
variable, ie. the real variable x is replaced by the complex variable z, it has 
solutions whose type of behaviour depends upon where z lies in the complex 


plane. For the particular case 4 = —1, when Stokes’ equation takes the form 
Wy 
ae 


and the two WKB solutions (with the inverse fourth root written explicitly) are 
A ea) 
yia(z) = ow exp | 327 ; (25.51) 


one of the solutions, Ai(z) (see section 25.6), has the property that it is real 
whenever z is real, whether positive or negative. For negative real z it has 
sinusoidal behaviour, but it becomes an evanescent wave for real positive z. 

Since the function z*/? has a branch point at z = 0 and therefore has an abrupt 
(complex) change in its argument there, it is clear that neither of the two functions 
in (25.51), nor any fixed combination of them, can be equal to Ai(z) for all values 
of z. More explicitly, for z real and positive, Ai(z) is proportional to y,(z), which 
is real and has the form of a decaying exponential function, whilst for z real and 
negative, when z*/* is purely imaginary and y;(z) and y2(z) are both oscillatory, 
it is clear that Ai(z) must contain both y; and y2 with equal amplitudes. 

The actual combinations of y;(z) and y2(z) needed to coincide with these two 
asymptotic forms of Ai(z) are as follows. 


For z real and > 0, ciyi(z) = a exp |-32""| é (25.52) 
For z real and < 0, ool yy (z)e™/4 — yo(z)e"/4 J 
1 ee ee 1 
= Tata sin | 5 zpPr4 ii : (25.53) 


Therefore it must be the case that the constants used to form Ai(z) from 
the solutions (25.51) change as z moves from one part of the complex plane to 
another. In fact, the changes occur for particular values of the argument of z; 
these boundaries are therefore radial lines in the complex plane and are known 
as Stokes lines. For Stokes’ equation they occur when arg z is equal to 0, 27/3 or 
4x /3. 

The general occurrence of a change in the arbitrary constants used to make 
up a solution, as its argument crosses certain boundaries in the complex plane, is 
known as the Stokes phenomenon and is discussed further in subsection 25.7.4. 
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> Apply the WKB method to the problem of finding the quantum energy levels E of a 
particle of mass m bound in a symmetrical one-dimensional potential well V(x) that has 
only a single minimum. The relevant Schrédinger equation is 


Relate the problem close to each of the classical ‘turning points’, x = +a at which E — 
V(x) = 0, to Stokes’ equation and assume that it is appropriate to use the solution Ai(x) 
given in equations (25.52) and (25.53) at x =a. Show that if the general WKB solution in 
the ‘classically allowed’ region —a < x <a is to match such Airy solutions at both turning 
points, then 


is k(x) dx = (n+ 5)n, 


a 


where k?(x) = 2m[ E— V(x) ]/h? and n=0,1,2,.... 

For a symmetric potential V(x) = Vox, where s is a positive integer, show that in this 
approximation the energy of the nth level is given by E, = c,(n + $)?/*, where c, is a 
constant depending on s but not upon n. 


We start by multiplying the equation through by 2m/h?, writing 2m[ E—V(x)]/h? as k?(x), 
and rearranging the equation to read 
? 
LY + ke(x)y = 0, (25.54) 
dx? 


noting that, with E and V(x) given, the equation E = V(a) determines the value of a and 
that k(a) = 0. 
For —a < x < a, where k*(x) is positive, the form of the WKB solutions are given 


directly by (25.49) as 
C x 
= tif k(u)dul. 
pe 263) exp +i f (u) u| 


Just beyond the turning point x = a, where 


E—V(x) =0—V'(a\(x — a) + O[(x— a)", 


equation (25.54) can be approximated by 
yp  22mvV'(a) 


ae” OR 
This, in turn, can be reduced to Stokes’ equation by first setting x—a = wz and p(x) = y(z), 
so converting it into 


(x —a)p = 0. (25.55) 


1@y  2umV'(a) 


pee oe bee 
and then choosing p = [h?/2mV"(a)]'°. The equation then reads 
ay 
a 


Since the solution must be evanescent for x > a, ie. for z > 0, we assume that the 
appropriate solution there is Ai(z); this implies that, for z small and negative (just inside 
the classically allowed region), the solution has the form given by (25.53), namely 


A Pau ee? 372, & 
(ayia sin] 5 Zz) ber : 
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for some constant A. This form is only valid for negative z close to z = 0 and is not 
appropriate within the well as a whole, where the approximation (25.55) leading to Stokes’ 
equation is not valid. However, it does allow us to determine the correct combination of 
the WKB solutions found earlier for the proper continuation inside the well of the solution 


found for z > 0. This is 
wi(x) = = sin (/ k(u) du + *) , 


A similar argument gives the continuation inside the well of the evanescent solution 
required in the region x < —a as 


r(x) = cc sin (/. k(u) du+ i) ; 


However, for a consistent solution to the problem, these two functions must match, both 
in magnitude and slope, at any arbitrary point x inside the well.We therefore require both 
of the equalities 


ae sin (/ k(u) du + i) = Tae sin (| Hendu + ;) (i) 


and 


é e Tl A a rt 
= TES sin (/ k(u) du + 5) + 65) [—k(x)] cos (/ k(u) du + i) 


1 Bk, ‘3 1 B bs T d 
— "3B sin ([ kenau+ 2) + 269) [k(x)] cos ([ kunau+ 3) . (ii) 


The general condition for the validity of the WKB solutions is that the derivatives of the 
function appearing in the phase integral are small in some sense (see subsection 25.7.3 for 
a more general discussion); here, if k’//k3 < k/J/k, ie. k’ < k?, then we can ignore the 
k’ terms in equation (ii) above. In fact, for this particular situation, this approximation 
is not needed since the first of the equalities, equation (i), ensures that the k’-dependent 
terms in the second equality (ii) cancel. Either way, we are left with a pair of homogeneous 
equations for A and B. For them to give consistent values for the ratio A/B, it must be 
that 
A 


; ‘ 1 B . 7 
763) sin (/ k(u) du + *) x 703) [k(x)] cos ([ kiran 2) 
7 ie L=ceaiicee ([ kau ~ *) x = sin (| Hond 4) 


This condition reduces to 


sin] (kn du $) + (f 


=e 
= 
= 
na 
= 
| 
T 
Nila Bla 
Nie cae 
us 
ll 
i) 


i ee ol 
us 
ll 
i) 


@. 
=} 
Poa 
— 
3 
= 
= 
= 
= 
nu 
° 
| 
T 


=> k(u) du = (n+ 5)n. 
Since k(x) > 0 in the range —a < x < a, n may take the values 0, 1, 2,.... 
If V(x) has the form V(x) = Vox?’ then, for the nth allowed energy level, E, = Voa?’ 
and 
2m 


P(x) == 
w= 


(En _ Vox"). 
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The result just proved gives 


an 2. Vi 
Y V - o (a25 — x?°)"/? dx = (n+ 5). 
cna) 


Writing x = va, shows that the integral is proportional to a‘+!I,, where I, is the integral 
between —1 and +1 of (1 —v%)!/? and does not depend upon n. Thus E, o a?’ and 
as*! oc (n+ 4), implying that E, oc (n + $)*/+1, 

Although not asked for, we note that the above result indicates that, for a simple 
harmonic oscillator, for which s = 1, the energy levels [E, ~ (n+ 3)] are equally spaced, 
whilst for very large s, corresponding to a square well, the energy levels vary as n?. Both 
of these results agree with what is found from detailed analyses of the individual cases. < 


25.7.3 Accuracy of the WKB solutions 


We may also ask when we can expect the WKB solutions to the Stokes’ equation 
to be reasonable approximations. Although our final form for the WKB solutions 
is not exactly that used when the condition |n'ky'| « |n?| was derived, it should 
give the same order of magnitude restriction as a more careful analysis. For the 
derivation of (25.51), kj = —1, n(z) = [f(z)]!” =z!”, and the criterion becomes 
5|z~'/?| < |z|, or, in round terms, |z|? > 1. 

For the more general equation, typified by (25.42), the condition for the validity 
of the WKB solutions can usually be satisfied by making some quantity, often |z], 
sufficiently large. Alternatively, a parameter such as ko can be made large enough 
that the validity criterion is satisfied to any pre-specified level. However, from a 
practical point of view, natural physical parameters cannot be varied at will, and 
requiring z to be large may well reduce the value of the method to virtually zero. 
It is normally more useful to try to obtain an improvement on a WKB solution 
by multiplying it by a series whose terms contain increasing inverse powers of 
the variable, so that the result can be applied successfully for moderate, and not 
just excessively large, values of the variable. 

We do not have the space to discuss the properties and pitfalls of such 
asymptotic expansions in any detail, but exercise 25.18 will provide the reader 
with a model of the general procedure. A few particular points that should be 
noted are given as follows. 


(i) If the multiplier is analytic as z — oo, then it will be represented by a 
series that is convergent for |z| greater than some radius of convergence 
R. 

(ii) If the multiplier is not analytic as z > ©, as is usually the case, then the 
multiplier series eventually diverges and there is a z-dependent optimal 
number of terms that the series should contain in order to give the best 
accuracy. 

(iii) For a fixed value of argz, the asymptotic expansion of the multiplier is 
unique. However, the same asymptotic expansion can represent more than 
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one function and the same function may need different expansions for 
different values of arg z. 


Finally in this subsection we note that, although the form of equation (25.42) may 
appear rather restrictive, in that it contains no term in y’, the results obtained so 
far can be applied to an equation such as 

Wy 


| 
a Pee + Q(z)y =0. (25.56) 


To make this possible, a change of either the dependent or the independent 
variable is made. For the former we write 


Z 2 
vie)=erexp (5 | Pend) = or + (o-4P?-52) v=o 


whilst for the latter we introduce a new independent variable ¢ defined by 


dl ’ ay ge\> > 
+ =exp(- / Piutdu) => +0() y=0. 


In either case, equation (25.56) is reduced to the form of (25.42), though it 
will be clear that the two sets of WKB solutions (which are, of course, only 
approximations) will not be the same. 


25.7.4 The Stokes phenomenon 


As we saw in subsection 25.7.2, the combination of WKB solutions of a differential 
equation required to reproduce the asymptotic form of the accurate solution y(z) 
of the same equation, varies according to the region of the z-plane in which z lies. 
We now consider this behaviour, known as the Stokes phenomenon, in a little 
more detail. 

Let yi(z) and y2(z) be the two WKB solutions of a second-order differential 
equation. Then any solution Y (z) of the same equation can be written asymptot- 
ically as 


Y(z) ~ Aiyi(z) + A2ya(z), (25.57) 


where, although we will be considering (abrupt) changes in them, we will continue 
to refer to A; and A» as constants, as they are within any one region. In order to 
produce the required change in the linear combination, as we pass over a Stokes 
line from one region of the z-plane to another, one of the constants must change 
(relative to the other) as the border between the regions is crossed. 

At first sight, this may seem impossible without causing a discernible discon- 
tinuity in the representation of Y(z). However, we must recall that the WKB 
solutions are approximations, and that, as they contain a phase integral, for 
certain values of argz the phase ¢(z) will be purely imaginary and the factors 
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exp[+ id(z)] will be purely real. What is more, one such factor, known as the 
dominant term, will be exponentially large, whilst the other (the subdominant term) 
will be exponentially small. A Stokes line is precisely where this happens. 

We can now see how the change takes place without an observable discontinuity 
occurring. Suppose that y;(z) is very large and y2(z) is very small on a Stokes line. 
Then a finite change in A> will have a negligible effect on Y(z); in fact, Stokes 
showed, for some particular cases, that the change is less than the uncertainty 
in y;(z) arising from the approximations made in deriving it. Since the solution 
with any particular asymptotic form is determined in a region bounded by two 
Stokes lines to within an overall multiplicative constant and the original equation 
is linear, the change in Ay when one of the Stokes lines is crossed must be 
proportional to Aj, i.e. Ay changes to Az +SAj, where S is a constant (the Stokes 
constant) characteristic of the particular line but independent of A; and Ap. It 
should be emphasised that, at a Stokes line, if the dominant term is not present 
in a solution, then the multiplicative constant in the subdominant term cannot 
change as the line is crossed. 

As an example, consider the Bessel function Jo(z) of zero order. It is single- 
valued, differentiable everywhere, and can be written as a series in powers of z?. It 
is therefore an integral even function of z. However, its asymptotic approximations 
for two regions of the z-plane, Re z > 0 and z real and negative, are given by 


1 1 Pe wit 
Jo(2) ~~ ae re (ee 4 + eel) | | arg(z)| < $a, larg(e!)| < fx, 


Jin 


Jo(z) (een + ete) , arg(z) = 7, arg(z!/?) = —5n. 


aes | 

© Von 
We note in passing that neither of these expressions is naturally single-valued, 
and a prescription for taking the square root has to be given. Equally, neither is 
an even function of z. For our present purpose the important point to note is 
that, for both expressions, on the line argz = 2/2 both z-dependent exponents 
become real. For large |z| the second term in each expression is large; this is the 
dominant term, and its multiplying constant e'"/4 is the same in both expressions. 
Contrarywise, the first term in each expression is small, and its multiplying 
constant does change, from e~'7/* to e3*/4, as argz passes through 2/2 whilst 
increasing from 0 to z. It is straightforward to calculate the Stokes constant for 
this Stokes line as follows: 


_ Ax(new) — Az(old) _ ein/s _ @—in/4 — ini 


—in/2 _»4; 

S A oin/ e 2i. 
If we had moved (in the negative sense) from argz = 0 to argz = —1, the relevant 
Stokes line would have been arg z = —7/2. There the first term in each expression 


is dominant, and it would have been the constant e'"/* in the second term that 


would have changed. The final argument of z~!/? would have been +7/2. 
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Finally, we should mention that the lines in the z-plane on which the expo- 
nents in the WKB solutions are purely imaginary, and the two solutions have 
equal amplitudes, are usually called the anti-Stokes lines. For the general Bessel’s 
equation they are the real positive and real negative axes. 


25.8 Approximations to integrals 


In this section we will investigate a method of finding approximations to the 
values or forms of certain types of infinite integrals. The class of integrals to 
be considered is that containing integrands that are, or can be, represented by 
exponential functions of the general form g(z) exp[ f(z) ]. The exponents f(z) may 
be complex, and so integrals of sinusoids can be handled as well as those with 
more obvious exponential properties. We will be using the analyticity properties 
of the functions of a complex variable to move the integration path to a part 
of the complex plane where a general integrand can be approximated well by a 
standard form; the standard form is then integrated explicitly. 

The particular standard form to be employed is that of a Gausssian function of 
a real variable, for which the integral between infinite limits is well known. This 
form will be generated by expressing f(z) as a Taylor series expansion about a 
point zo, at which the linear term in the expansion vanishes, i.e. where f'(z) = 0. 
Then, apart from a constant multiplier, the exponential function will behave like 
exp[ 5h" (Zo)l(z —z9)?] and, by choosing an appropriate direction for the contour 
to take as it passes through the point, this can be made into a normal Gaussian 
function of a real variable and its integral may then be found. 


25.8.1 Level lines and saddle points 


Before we can discuss the method outlined above in more detail, a number of 
observations about functions of a complex variable and, in particular, about the 
properties of the exponential function need to be made. For a general analytic 
function, 


F(Z) = Gy) + ip(x, y), (25.58) 


of the complex variable z = x + iy, we recall that, not only do both ¢ and p 
satisfy Laplace’s equation, but V@ and Vy are orthogonal. This means that the 
lines on which one of ¢ and yp is constant are exactly the lines on which the other 
is changing most rapidly. 

Let us apply these observations to the function 


h(z) = expL f(z)] = exp() exp(iyp), (25.59) 


recalling that the functions ¢ and w are themselves real. The magnitude of h(z), 
given by exp(@), is constant on the lines of constant ¢, which are known as the 
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Figure 25.12 A greyscale plot with associated contours of the value of |h(z)|, 
where h(z) = exp[i(z? + 62? — 15z + 8)], in the neighbourhood of one of its 
saddle points; darker shading corresponds to larger magnitudes. The plot also 
shows the two level lines (thick solid lines) through the saddle and part of the 
line of steepest descents (dashed line) passing over it. At the saddle point, the 
angle between the line of steepest descents and a level line is 7/4. 


level lines of the function. It follows that the direction in which the magnitude 
of h(z) changes most rapidly at any point z is in a direction perpendicular to 
the level line passing through that point. This is therefore the line through z on 
which the phase of h(z), namely y(z), is constant. Lines of constant phase are 
therefore sometimes referred to as lines of steepest descent (or steepest ascent). 

We further note that |h(z)| can never be negative and that neither @ nor p 
can have a finite maximum at any point at which f(z) is analytic. This latter 
observation follows from the fact that at a maximum of, say, (x, y), both 67/0x* 
and 07/dy* would have to be negative; if this were so, Laplace’s equation could 
not be satisfied, leading to a contradiction. A similar argument shows that a 
minimum of either ¢ or w is not possible wherever f(z) is analytic. A more 
positive conclusion is that, since the two unmixed second partial derivatives 
0/0x* and 6?/éy? must have opposite signs, the only possible conclusion 
about a point at which V@ is defined and equal to zero is that the point is a 
saddle point of h(z). An example of a saddle point is shown as a greyscale plot 
in figure 25.12 and, more pictorially, in figure 5.2. 
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From the observations contained in the two previous paragraphs, we deduce 
that a path that follows the lines of steepest descent (or ascent) can never form 
a closed loop. On such a path, ¢, and hence |h(z)|, must continue to decrease 
(increase) until the path meets a singularity of f(z). It also follows that if a level 
line of h(z) forms a closed loop in the complex plane, then the loop must enclose 
a singularity of f(z). This may (if ¢ — oo) or may not (if ¢ — —oo) produce a 
singularity in h(z). 

We now turn to the study of the behaviour of h(z) at a saddle point and how 
this enables us to find an approximation to the integral of h(z) along a contour 
that can be deformed to pass through the saddle point. At a saddle point zo, at 
which f’(zo) = 0, both V@ and Vy are zero, and consequently the magnitude and 
phase of h(z) are both stationary. The Taylor expansion of f(z) at such a point 
takes the form 


F(z) = f(Zo) +0 + icone — 2) + O(z — 20)’. (25.60) 


We assume that f”(zo) # 0 and write it explicitly as f”(zo) = Ae”, thus defining 
the real quantities A and «. If it happens that f”(zo) = 0, then two or more 
saddle points coalesce and the Taylor expansion must be continued until the first 
non-vanishing term is reached; we will not consider this case further, though the 
general method of proceeding will be apparent from what follows. If we also 
abbreviate the (in general) complex quantity f(zo) to fo, then (25.60) takes the 
form 


f(z) = fo + 4Ae(z — 20)? + O( — 20). (25.61) 


To study the implications of this approximation for h(z), we write z — zo as 
pe” with p and @ both real. Then 


|h(z)| = |exp(fo)| expl £4p? cos(20 + x) + O(p')]. (25.62) 


This shows that there are four values of 0 for which |h(z)| is independent of p (to 
second order). These therefore correspond to two crossing level lines given by 


0=4(43n—«4) and 0=5(+3n—a). (25.63) 


The two level lines cross at right angles to each other. It should be noted that 
the continuations of the two level lines away from the saddle are not straight 
in general. At the saddle they have to satisfy (25.63), but away from it the lines 
must take whatever directions are needed to make V¢ = 0. In figure 25.12 one of 
the level lines (|h| = 1) has a continuation (y = 0) that is straight; the other does 
not and bends away from its initial direction x = 1. 

So far as the phase of h(z) is concerned, we have 


arg[h(z)] = arg(fo) + 4Ap? sin(20 + x) + O(p3), 


which shows that there are four other directions (two lines crossing at right 
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angles) in which the phase of h(z) is independent of p. They make angles of 2/4 
with the level lines through zo and are given by 


d= 50, 0 3(tn a), O=n7 


a. 


From our previous discussion it follows that these four directions will be the 
lines of steepest descent (or ascent) on moving away from the saddle point. In 
particular, the two directions for which the term cos(20 + «) in (25.62) is negative 
will be the directions in which |h(z)| decreases most rapidly from its value at the 
saddle point. These two directions are antiparallel, and a steepest descents path 
following them is a smooth locally straight line passing the saddle point. It is 
known as the line of steepest descents (1.s.d.) through the saddle point. Note that 
‘descents’ is plural as on this line the value of |h(z)| decreases on both sides of the 
saddle. This is the line which we will make the path of the contour integral of 
h(z) follow. Part of a typical l.s.d. is indicated by the dashed line in figure 25.12. 


25.8.2 Steepest descents method 


To help understand how an integral along the line of steepest descents can be 
handled in a mechanical way, it is instructive to consider the case where the 
function f(z) = —Bz? and h(z) = exp(—fz?). The saddle point is situated at 
z =z = 0, with fo = f(z) = 1 and f”(zo) = —2f, implying that A = 2/f| and 
a =+n+argf, with the + sign chosen to put « in the range 0 < a < 27. Then 
the l.s.d. is determined by the requirement that sin(20+«) = 0 whilst cos(20+«) is 
negative; together these imply that, for the l.s.d., 0 = —4 arg B or 0 =1— 5 arg f. 

Since the Taylor series for f(z) = —Bz? terminates after three terms, expansion 
(25.61) for this particular function is not an approximation to h(z), but is exact. 
Consequently, a contour integral starting and endingin regions of the complex 
plane where the function tends to zero and following the |.s.d. through the saddle 
point at z = 0 will not only have a straight-line path, but will yield an exact 
result. Setting z = te~7*8P will reduce the integral to that of a Gaussian function: 


e@) 
al oll dp — o-barek | 
ime IB| 


The saddle-point method for a more general function aims to simulate this 
approach by deforming the integration contour C and forcing it to pass through 
a saddle point z = zo, where, whatever the function, the leading z-dependent term 
in the exponent will be a quadratic function of z — zo, thus turning the integrand 
into one that can be approximated by a Gaussian. 

The path well away from the saddle point may be changed in any convenient 
way so long as it remains within the relevant sectors, as determined by the end- 
points of C. By a ‘sector’ we mean a region of the complex plane, any part of 
which can be reached from any other part of the same region without crossing 
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any of the continuations to infinity of the level lines that pass through the saddle. 
In practical applications the start- and end-points of the path are nearly always 
at singularities of f(z) with Re f(z) — —oo and |h(z)| > 0. 

We now set out the complete procedure for the simplest form of integral 
evaluation that uses a method of steepest descents. Extensions, such as including 
higher terms in the Taylor expansion or having to pass through more than one 
saddle point in order to have appropriate termination points for the contour, can 
be incorporated, but the resulting calculations tend to be long and complicated, 
and we do not have space to pursue them in a book such as this one. 

As our general integrand we take a function of the form g(z)h(z), where, as 
before, h(z) = exp[f(z)]. The function g(z) should neither vary rapidly nor have 
zeros or singularities close to any saddle point used to evaluate the integral. 
Rapidly varying factors should be incorporated in the exponent, usually in the 
form of a logarithm. Provided g(z) satisfies these criteria, it is sufficient to treat 
it as a constant multiplier when integrating, assigning to it its value at the saddle 
point, g(zo). 

Incorporating this and retaining only the first two non-vanishing terms in 
equation (25.61) gives the integrand as 


g(Zo) exp(fo) exp[ 5Ae” (z — 20)’]. (25.64) 


From the way in which it was defined, it follows that on the l.s.d. the imaginary 
part of f(z) is constant (= Im fo) and that the final exponent in (25.64) is either 
zero (at zo) or negative. We can therefore write it as —s?, where s is real. Further, 
since exp[f(z)] — 0 at the start- and end-points of the contour, we must have 
that s runs from —oo to +00, the sense of s being chosen so that it is negative 
approaching the saddle and positive when leaving it. 

Making this change of variable, 


/2 
5Ael(z —z)? =—s*, with dz =+ A exp[ si(x —a)|ds, 
(25.65) 


allows us to express the contribution to the integral from the neighbourhood of 
the saddle point as 


+8(z0) exptfon | expl 4i(n — 2) ] i ~_exp(—s?) ds: 


—o0 
The simple saddle-point approximation assumes that this is the only contribution, 
and gives as the value of the contour integral 


2 
[ svexpL fe] dz = ty) 5 alco) explo) expl fla — 2), 
c (25.66) 


where we have used the standard result that ies exp(—s*) ds = Jt The overall + 
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sign is determined by the direction @ in the complex plane in which the distorted 
contour passes through the saddle point. If —51 <0< 50, then the positive 
sign is taken; if not, then the negative sign is appropriate. In broad terms, if the 
integration path through the saddle is in the direction of an increasing real part 
for z, then the overall sign is positive. 

Formula (25.66) is the main result from a steepest descents approach to evalu- 
ating a contour integral of the type considered, in the sense that it is the leading 
term in any more refined calculation of the same integral. As can be seen, it 
is as an ‘omnibus’ formula, the various components of which can be found by 
considering a number of separate, less-complicated, calculations. 

Before presenting a worked example that generates a substantial result, useful 
in another connection, it is instructive to consider an integral that can be simply 
and exactly evaluated by other means and then apply the saddle-point result to 
it. Of course, the steepest descents method will appear heavy-handed, but our 
purpose is to show it in action and to try to see why it works. 

Consider the real integral 


I -| exp(10t — 1”) dt. 


This can be evaluated directly by making the substitution s = t — 5 as follows: 


I= / : exp(10t — t*) dt = / : exp(25 — s”)ds = ef : exp(—s*) ds = me”. 


The saddle-point approach to the same problem is to consider the integral as a 
contour integral in the complex plane, but one that lies along the real axis. The 
saddle points of the integrand occur where f’(t) = 10 — 2t = 0; there is thus a 
single saddle point at t = to = 5. This is on the real axis, and no distortion of the 
contour is necessary. The value fo of the exponent is f(5) = 50 — 25 = 25, whilst 
its second derivative at the saddle point is f’(5) = —2. Thus, A = 2 and a= 7. 
The contour clearly passes through the saddle point in the direction 0 = 0, i.e. in 
the positive sense on the real axis, and so the overall sign must be +. Since g(to) 
is formally unity, we have all the ingredients needed for substitution in formula 
(25.66), which reads 


l=+4/ = 1 exp(25) exp[ Si(x — 2)] = fre”. 


As it happens, this is exactly the same result as that obtained by accurate 
calculation. This would not normally be the case, but here it is, because of the 
quadratic nature of 10¢—1?; all of its derivatives beyond the second are identically 
zero and no approximation of the exponent is involved. 

Given the very large value of the integrand at the saddle point itself, the 
reader may wonder whether there really is a saddle there. However, evaluating 
the integrand at points lying on a line through the saddle point perpendicular 
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to the Ls.d., Le. on the imaginary t-axis, provides some reassurance. Whether p is 
positive or negative, 


A(S + ig) = exp(50 + 10in — 25 — 10iu + p?) = exp(25 +p’). 


This is greater than h(5) for all 4 and increases as |u| increases, showing that 
the integration path really does lie at a minimum of h(t) for a traversal in this 
direction. 

We now give a fully worked solution to a problem that could not be easily 
tackled by elementary means. 


> Apply the saddle-point method to the function defined by 


F(x) = a cos(4s* + xs) ds 
T Jo 


to show that its form for large positive real x is one that tends asymptotically to zero, hence 
enabling F(x) to be identified with the Airy function, Ai(x). 


We first express the integral as an exponential function and then make the change of 
variable s = x!/*t to bring it into the canonical form J g(t)exp[f(t)] dt as follows: 


Fix) == [ costs? + x9)ds 
0 


1 00 ; 
= =| exp[i(4s* + xs) ] ds 


—« 
1 f? ap 3/2/43 
=— x explix’ (st + t)] dt. 
Qi: Jue 
We now seek to find an approximate expression for this contour integral by deforming its 
path along the real t-axis into one passing over a saddle point of the integrand. Considered 
as a function of t, the multiplying factor x!/?/2z is a constant, and any effects due to the 
proximity of its zeros and singularities to any saddle point do not arise. 
The saddle points are situated where 


0O=f() =r??? 41) => t=Hi. 
For reasons discussed later, we choose to use the saddle point at t = fo =i. At this point, 


f= x?(—4i +i= —3x3/?? and Ae” = f"(i) = ix*/?(2i) 2x3/2, 


and so A = 2x?/? and «=z. 
Now, expanding f(t) around t =i by setting t=i+ pe”, we have 


SO=fY+O0+ a i"ie— i +O[(t—i)*] 


a aise + : 2x3)? et 92.0210 + O(p?). 
For the |.s.d. contour that crosses the saddle point we need the second term in this last 
line to decrease as p increases. This happens if x + 20 = +7, ie. if 0 = 0 or 06 = —2x (or 
+7); thus, the Ls.d. through the saddle is orientated parallel to the real t-axis. Given the 
initial contour direction, the deformed contour should approach the saddle point from the 
direction 06 = —z and leave it along the line 0 = 0. Since —z/2 < 0 < 2/2, the overall sign 
of the ‘omnibus’ approximation formula is determined as positive. 
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Finally, putting the various values into the formula yields 


2n\ 1? 
F(x) ~ + (4) ati) expl f(i) Jexpl 4a — 9) 


In \M? x12 2 : 


1 2 
= Dax exp (-3 7) : 
This is the leading term in the asymptotic expansion of F(x), which, as shown in equation 
(25.39), is a particular contour integral solution of Stokes’ equation. The fact that it tends 
to zero in a monotonic way as x > +00 allows it to be identified with the Airy function, 
Ai(x). 

We may ask why the saddle point at t = —i was not used. The answer to this is as 
follows. Of course, any path that starts and ends in the right sectors will suffice, but 
if another saddle point exists close to the one used, then the Taylor expansion actually 
employed is likely to be less effective than if there were no other saddle points or if there 
were only distant ones. 

An investigation of the same form as that used at t = +i shows that the saddle at t = —i 
is higher by a factor of exp(4x*/?) and that its l.s.d. is orientated parallel to the imaginary 
t-axis. Thus a path that went through it would need to go via a region of largish negative 
imaginary t, over the saddle at t = —i, and then, when it reached the col at t = +i, bend 
sharply and follow part of the same |.s.d. as considered earlier. Thus the contribution from 
the t = —i saddle would be incomplete and roughly half of that from the t = +i saddle 
would still have to be included. The more serious error would come from the first of these, 
as, Clearly, the part of the path that lies in the plane Re t = 0 is not symmetric and is far 
from Guassian-like on the side nearer the origin. The Gaussian-path approximation used 
will therefore not be a good one, and, what is more, the resulting error will be magnified by 
a factor exp($x*/) compared with the best estimate. So, both on the grounds of simplicity 
and because the effect of the other (neglected) saddle point is likely to be less severe, we 
choose to use the one at t = +i. < 


25.8.3 Stationary phase method 


In the previous subsection we showed how to use the saddle points of an 
exponential function of a complex variable to evaluate approximately a contour 
integral of that function. This was done by following the lines of steepest descent 
that passed through the saddle point; these are lines on which the phase of the 
exponential is constant but its amplitude is varying at the maximum possible 
rate for that function. We now introduce an alternative method, one that entirely 
reverses the roles of amplitude and phase. To see how such an alternative approach 
might work, it is useful to study how the integral of an exponential function of 
a complex variablecan be represented as the sum of infinitesimal vectors in the 
complex plane. 
We start by studying the familiar integral 


In = / exp(—z’) dz, (25.67) 


‘00 
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which we already know has the value ,/x when z is real. This choice of demon- 
stration model is not accidental, but is motivated by the fact that, as we have 
already shown, in the neighbourhood of a saddle point all exponential integrands 
can be approximated by a Gaussian function of this form. 

The same integral can also be thought of as an integral in the complex plane, 
in which the integration contour happens to be along the real axis. Since the 
integrand is analytic, the contour could be distorted into any other that had the 
same end-points, z = —oo and z = +, both on the real axis. 

As a particular possibility, we consider an arc of a circle of radius R centred 
on z = 0. It is easily shown that cos20 > 1+ 40/z for —2/4 < 0 < 0, where 0 is 
measured from the positive real z-axis and —a < 0 < a. It follows from writing 
z = Re on the arc that, if the arc is confined to the region —2/4 < 0 < 0 
(actually, |0| < 1/4 is sufficient), then the integral of exp(—z*) tends to zero as 
R — o anywhere on the arc. A similar result holds for an arc confined to the 
region ||0| — z| < 2/4. We also note for future use that, for 1/4 < @ < 32/4 
or —1/4 > 0 > —3n/4, the integrand exp(—z?) grows without limit as R > 00, 
and that the larger R is, the more precipitous is the ‘drop or rise’ in its value on 
crossing the four radial lines 0 = +7/4 and 0 = +37/4. 

Now consider a contour that consists of an arc at infinity running from 0 = 2 
to 0 = x —« joined to a straight line, 0 = —a, which passes through z = 0 and 
continues to infinity, where it in turn joins an arc at infinity running from 0 = —« 
to 0 = 0. This contour has the same start- and end-points as that used in Io, 
and so the integral of exp(—z?) along it must also have the value ,/m. As the 
contributions to the integral from the arcs vanish, provided « < 7/4, it follows 
that the integral of exp(—z?) along the infinite line 0 = —a is ./. If we now take 
a arbitrarily close to 2/4, we may substitute z = sexp(—iz/4) into (25.67) and 
obtain 


ive 


Jr = : exp(—z’) dz 


= exp(—im/4) Le exp(is”) ds (25.68) 


—0oo 


= ,/27 exp(—iz/4) | [ vostgau?yau i f sin( nu?) | (25.69) 
0 0 


The final line was obtained by making a scale change s = \/z/2u. This enables 
the two integrals to be identified with the Fresnel integrals C(x) and S(x), 


x x 
C(x) -| cos(5mu") du and S(x) -| sin(;7u") du, 
0 0 
mentioned on page 645. Equation (25.69) can be rewritten as 


Ct = 2n[C(w) + iS(o)], 


2 
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from which it follows that C(co) = S(#) = 5. Clearly, C(—o) = S(—o) = 5. 

We are now in a position to examine these two equivalent ways of evaluating I 
in terms of sums of infinitesmal vectors in the complex plane. When the integral 
fe exp(—z?) dz is evaluated as a real integral, or a complex one along the real 
z-axis, each element dz generates a vector of length exp(—z?)dz in an Argand 
diagram, usually called the amplitude-phase diagram for the integral. For this 
integration, whilst all vector contributions lie along the real axis, they do differ in 
magnitude, starting vanishingly small, growing to a maximum length of 1 x dz, 
and then reducing until they are again vanishingly small. At any stage, their 
vector sum (in this case, the same as their algebraic sum) is a measure of the 
indefinite integral 


x 
I(x) = / exp(—z’) dz. (25.70) 
The total length of the vector sum when x — 0 is, of course, Jt and it should 
not be overlooked that the sum is a vector parallel to (actually coinciding with) 
the real axis in the amplitude—phase diagram. Formally this indicates that the 
integral is real. This ‘ordinary’ view of evaluating the integral generates the same 
amplitude—phase diagram as does the method of steepest descents. This is because 
for this particular integrand the l.s.d. never leaves the real axis. 

Now consider the same integral evaluated using the form of equation (25.69). 
Here, each contribution, as the integration variable goes from u to u + du, is of 
the form 


g(u) du = cos($7u") dut+i sin($7u*) du. 


As infinitesimal vectors in the amplitude—phase diagram, all g(u) du have the same 
magnitude du, but their directions change continuously. Near u = 0, where u? 
is small, the change is slow and each vector element is approximately equal to 
»/2n exp(—in/4) du; these contributions are all in phase and add up to a significant 
vector contribution in the direction 0 = —7/4. This is illustrated by the central 
part of the curve in part (b) of figure 25.13, in which the amplitude—phase diagram 
for the ‘ordinary’ integration, discussed above, is drawn as part (a). 

Part (b) of the figure also shows that the vector representing the indefinite 
integral (25.70) initially (s large and negative) spirals out, in a clockwise sense, 
from around the point 0+ i0 in the amplitude—phase diagram and ultimately (s 
large and positive) spirals in, in an anticlockwise direction, to the point \/x + i0. 
The total curve is called a Cornu spiral. In physical applications, such as the 
diffraction of light at a straight edge, the relevant limits of integration are 
typically —oo and some finite value x. Then, as can be seen, the resulting vector 
sum is complex in general, with its magnitude (the distance from 0 + i0 to the 
point on the spiral corresponding to z = x) growing steadily for x < 0 but 
showing oscillations when x > 0. 
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e a a e 


(a) 


Figure 25.13 Amplitude-phase diagrams for the integral fe exp(—z?) dz 
using different contours in the complex z-plane. (a) Using the real axis, as in 
the steepest descents method. (b) Using the level line z = u exp(— in) that 
passes through the saddle point, as in the stationary phase method. (c) Using 
a path that makes a positive angle B (< 2/4) with the z-axis. 


The final curve, 25.13(c), shows the amplitude-phase diagram corresponding to 
an integration path that is along a line making a positive angle f (0 < B < 1/4) 
with the real z-axis. In this case, the constituent infinitesimal vectors vary in both 
length and direction. Note that the curve passes through its centre point with the 
positive gradient tan f and that the directions of the spirals around the winding 
points are reversed as compared with case (b). 

It is important to recognise that, although the three paths illustrated (and the 
infinity of other similar paths not illustrated) each produce a different phase— 
amplitude diagram, the vectors joining the initial and final points in the diagrams 
are all the same. For this particular integrand they are all (i) parallel to the 
positive real axis, showing that the integral is real and giving its sign, and (ii) of 
length \/z, giving its magnitude. 

What is apparent from figure 25.13(b), is that, because of the rapidly varying 
phase at either end of the spiral, the contributions from the infinitesimal vectors 
in those regions largely cancel each other. It is only in the central part of the 
spiral where the individual contributions are all nearly in phase that a substantial 
net contribution arises. If, on this part of the contour, where the phase is virtually 
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stationary, the magnitude of any factor, g(z), multiplying the exponential function, 
exp[ f(z)] ~ exp[Ae(z —z0)*], is at least comparable to its magnitude elsewhere, 
then this result can be used to obtain an approximation to the value of the 
integral of h(z) = g(z)exp[f(z)]. This is the basis of the method of stationary 
phase. 

Returning to the behaviour of a function exp[f(z)] at one of its saddle points, 
we can now see how the considerations of the previous paragraphs can be applied 
there. We already know, from equation (25.62) and the discussion immediately 
following it, that in the equation 


h(z) © g(zo) exp(fo) exp{ 5Ap?[cos(20 + «) + isin(20 + «)]} 
(25.71) 


the second exponent is purely imaginary on a level line, and equal to zero at 
the saddle point itself. What is more, since Vy = 0 at the saddle, the phase is 
stationary there; on one level line it is a maximum and on the other it is a 
minimum. As there are two level lines through a saddle point, a path on which 
the amplitude of the integrand is constant could go straight on at the saddle 
point or it could turn through a right angle. For the moment we assume that it 
runs continuously through the saddle. 

On the level line for which the phase at the saddle point is a minimum, we can 
write the phase of h(z) as approximately 


arg g(zo) + Im fo +0”, 


where v is real, iv? = 5Ae!(z — 29)? and, as previously, Ae* = f”(zo). Then 


ein/4 dv = + [+ en dz, (25.72) 


leading to an approximation to the integral of 


[ards = eetzodexptto) [/ explio?)y/ 5 explitdx — $2) | do 


—oo 


A 
= + g(20) exp(fo)/m explin/4)\/ > exp[i(qm — 52)] 


=+ yee) exp(fo) exp[ 5i(n —a)]. (25.73) 


Result (25.68) was used to obtain the second line above. The + ambiguity is again 
resolved by the direction 0 of the contour; it is positive if —32/4 < 0 < 1/4; 
otherwise, it is negative. 

What we have ignored in obtaining result (25.73) is that we have integrated 
along a level line and that therefore the integrand has the same magnitude far 
from the saddle as it has at the saddle itself. This could be dismissed by referring 
to the fact that contributions to the integral from the ends of the Cornu spiral 
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are self-cancelling, as discussed previously. However, the ends of the contour must 
be in regions where the integrand is vanishingly small, and so at each end of 
the level line we need to add a further section of path that makes the contour 
terminate correctly. 

Fortunately, this can be done without adding to the value of the integral. This 
is because, as noted in the second paragraph following equation (25.67), far from 
the saddle the level line will be at a finite height up a ‘precipitous cliff’ that 
separates the region where the integrand grows without limit from the one where 
it tends to zero. To move down the cliff-face into the zero-level valley requires 
an ever smaller step the further we move away from the saddle; as the integrand 
is finite, the contribution to the integral is vanishingly small. In figure 25.12, this 
additional piece of path length might, for example, correspond to the infinitesimal 
move from a point on the large positive x-axis (where h(z) has value 1) to a point 
just above it (where h(z) ~ 0). 

Now that formula (25.73) has been justified, we may note that it is exactly 
the same as that for the method of steepest descents, equation (25.66). A similar 
calculation using the level line on which the phase is a maximum also reproduces 
the steepest-descents formula. It would appear that ‘all roads lead to Rome’. 
However, as we explain later, some roads are more difficult than others. Where 
a problem involves using more than one saddle point, if the steepest-descents 
approach is tractable, it will usually be the more straight forward to apply. 

Typical amplitude-phase diagrams for an integration along a level line that 
goes straight through the saddle are shown in parts (a) and (b) of figure 25.14. 
The value of the integral is given, in both magnitude and phase, by the vector 
v joining the initial to the final winding points and, of course, is the same in 
both cases. Part (a) corresponds to the case of the phase being a minimum at the 
saddle; the vector path crosses v at an angle of —z/4. When a path on which the 
phase at the saddle is a maximum is used, the Cornu spiral is as in part (b) of 
the figure; then the vector path crosses v at an angle of +7/4. As can be seen, 
the two spirals are mirror images of each other. 

Clearly a straight-through level line path will start and end in different zero- 
level valleys. For one that turns through a right angle at the saddle point, the 
end-point could be in a different valley (for a function such as exp(—z’), there is 
only one other) or in the same one. In the latter case the integral will give a zero 
value, unless a singularity of h(z) happens to have been enclosed by the contour. 
Parts (c) and (d) of figure 25.14 illustrate the phase-amplitude diagrams for these 
two cases. In (c) the path turns through a right angle (+7/2, as it happens) at the 
saddle point, but finishes up in a different valley from that in which it started. In 
(d) it also turns through a right angle but returns to the same valley, albeit close 
to the other precipice from that near its starting point. This makes no difference 
and the result is zero, the two half spirals in the diagram producing resultants 
that cancel. 
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(a) (b) 


(c) (d) 


Figure 25.14 Amplitude—phase diagrams for stationary phase integration. (a) 
Using a straight-through path on which the phase is a minimum. (b) Using 
a straight-through path on which the phase is a maximum. (c) Using a level 
line that turns through +7/2 at the saddle point but starts and finishes in 
different valleys. (d) Using a level line that turns through a right angle but 
finishes in the same valley as it started. In cases (a), (b) and (c) the integral 
value is represented by v (see text). In case (d) the integral has value zero. 


We do not have the space to consider cases with two or more saddle points, 
but even more care is needed with the stationary phase approach than when 
using the steepest-descents method. At a saddle point there is only one 1.s.d. but 
there are two level lines. If more than one saddle point is required to reach the 
appropriate end-point of an integration, or an intermediate zero-level valley has 
to be used, then care is needed in linking the corresponding level lines in such a 
way that the links do not make a significant, but unknown, contribution to the 
integral. Yet more complications can arise if a level line through one saddle point 
crosses a line of steepest ascent through a second saddle. 


We conclude this section with a worked example that has direct links to the 
two preceding sections of this chapter. 
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p> In the worked example in subsection 25.8.2 the function 


F(x) = ~[ cos(+s° + xs) ds (*) 


was shown to have the properties associated with the Airy function, Ai(x), when x > 0. Use 
the stationary phase method to show that, for x <0 and —x sufficiently large, 


=( ee], 


1 : 
Jia sin 3 


in accordance with equation (25.53) for Ai(z). 


F(x) ~ 


Since the cosine function is an even function and its argument in (*) is purely real, we 
may consider F(x) as the real part of 


1 ioe) : 
G(x) = ae ie exp[i(4s? + xs)]. 
This is of the standard form for a saddle-point approach with g(s) = 1/2 and f(s) = 
i(4s° +xs). The latter has f’(s) = 0 when s* = —x. Since x < 0 there are two saddle points 


at s = +,/—x and s ,/—x. These are both on the real axis separated by a distance 


If —x is sufficiently large, the Gaussian-like stationary phase integrals can be treated 
separately and their contributions simply added. In terms of a phase—amplitude diagram, 
the Cornu spiral from the first saddle will have effectively reached its final winding point 
before the spiral from the second saddle begins. The second spiral therefore takes the final 
point of the first as its starting point; the vector representing its net contribution need not 
be in the same direction as that arising from the first spiral, and in general it will not be. 

Near the saddle at s = +./—x the form of f(s) is, in the usual notation, 


f(s) = fo + 3Ae"(s — so? 


a x)? 5 ova (p el”)? 
= Fae + /—x el"? p?(cos 20 + isin 20). 


For the exponent to be purely imaginary requires sin20 = 0, implying that the level lines 
are given by 0 = 0, 2/2, a or 32/2. The same conclusions hold at the saddle at s = —./—x, 
which differs only in that the sign of fo is reversed and « = 32/2 rather than 2/2; exp(ix) 
is imaginary in both cases. Thus the obvious path is one that approaches both saddles 
from the direction @ = a and leaves them in the direction 0 = 0. As —32/4 < 0 < 2/4, 
the + choice is resolved as positive at both saddles. 

Next we calculate the approximate values of the integrals from equation (25.73). At 


s=+,/—x it is 
2n 1 2i 3/2 i T 
ab Wars [zo ae E («- 5) 


1 uf 2 
+ exp | (Fl xp? a 


The corresponding contribution from the saddle at s = —,/—x is 


ai aes Ab 2 58 ex 2 pum 
2 Jax In P13 ane 2) I 
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Ir 


B 


Figure 25.15 The inductor—-capacitor—resistor network for exercise 25.1. 


which can also be simplified, and gives 


1 (2 
WL exp |i (5 x)3/? 7) |: 


Adding the two contributions and taking the real part of the sum, though this is not 
necessary here because the sum is real anyway, we obtain 


2 2 
F(x) = 2 ay cos ( (—xy°? 3) 


1 ff 2 T 
Fax sin (5 x24 5) > 


in agreement with the asymptotic form given in (25.53). < 


25.1 


25.2 


25.9 Exercises 


In the method of complex impedances for a.c. circuits, an inductance L is 
represented by a complex impedance Z, = iwL and a capacitance C by Zc = 
1/(imC). Kirchhoff’s circuit laws, 


> I; =0 at a node and Se Zl, = ys V; around any closed loop, 
i i Jj 


are then applied as if the circuit were a d.c. one. 

Apply this method to the a.c. bridge connected as in figure 25.15 to show 
that if the resistance R is chosen as R = (L/C)!/? then the amplitude of the 
current, Ip, through it is independent of the angular frequency w of the applied 
a.c. voltage Vo e". 

Determine how the phase of I, relative to that of the voltage source, varies 
with the angular frequency w. 

A long straight fence made of conducting wire mesh separates two fields and 
stands one metre high. Sometimes, on fine days, there is a vertical electric field 
over flat open countryside. Well away from the fence the strength of the field is 
Eo. By considering the effect of the transformation w = (1—z7)!/? on the real and 
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25;3 


25.4 


25.5 


25.6 


25.7 


25.8 


imaginary z-axes, find the strengths of the field (a) at a point one metre directly 
above the fence, (b) at ground level one metre to the side of the fence, and (c) 
at a point that is level with the top of the fence but one metre to the side of it. 
What is the direction of the field in case (c)? 


For the function 
z+c 
ftey=n (225), 


where c is real, show that the real part u of f is constant on a circle of radius 
ccosech u centred on the point z = ccothu. Use this result to show that the 
electrical capacitance per unit length of two parallel cylinders of radii a, placed 
with their axes 2d apart, is proportional to [cosh”!(d/a)]~!. 

Find a complex potential in the z-plane appropriate to a physical situation in 
which the half-plane x > 0, y = 0 has zero potential and the half-plane x < 0, 
y =0 has potential V. 

By making the transformation w = a(z + z~!)/2, with a real and positive, find 
the electrostatic potential associated with the half-plane r > a, s = 0 and the 
half-plane r < —a, s = 0 at potentials 0 and V, respectively. 

By considering in turn the transformations 


Zs 3e(w + Ww!) and w=exp{, 


where z = x+iy, w=rexpi0, € = €+ in and c is a real positive constant, show 
that z = ccosh¢ maps the strip € > 0,0 < 9 < 2z, onto the whole z-plane. Which 
curves in the z-plane correspond to the lines € = constant and 1 = constant? 
Identify those corresponding to € = 0, 7 = 0 and y = 2z. 
The electric potential ¢ of a charged conducting strip -c <x <c, y = 0, 
satisfies 
@ ~ —kIn(x? + y?)'? for large values of (x? + y?)!/?, 


with ¢ constant on the strip. Show that @ = Re[—kcosh”'(z/c)] and that the 
magnitude of the electric field near the strip is k(c? — x?)~!/”. 
For the equation 827 +z+1=0: 


(a) show that all three roots lie between the circles |z| = 3/8 and |z| = 5/8; 

(b) find the approximate location of the real root, and hence deduce that the 
complex ones lie in the first and fourth quadrants and have moduli greater 
than 0.5. 


Use contour integration to answer the following questions about the complex 
zeros of a polynomial equation. 


(a) Prove that z® + 327 + 7z+5 has two zeros in the first quadrant. 
(b) Find in which quadrants the zeros of 2z7 + 7z? + 10z + 6 lie. Try to locate 
them. 


The following is a method of determining the number of zeros of an nth-degree 
polynomial f(z) inside the contour C given by |z| = R: 
(a) put z = R(1+it)/(1 — it), with t = tan(/2), in the range -o <t<o; 
(b) obtain f(z) as 
A(t) +iB(t) (1 + it)" | 
(1—it)” (1+ it)’ 
(c) it follows that arg f(z) = tan7!(B/A) +ntan“'t; 
(d) and that Ac[arg f(z)] = Ac[tan“!(B/A)] + nz; 
(e) determine Ac[tan~'(B/A)] by evaluating tan7!(B/A) at t = --oo and finding 
the discontinuities in B/A by inspection or using a sketch graph. 
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25.9 


25.10 


25.11 


25.12 


25.13 


25.14 


Then, by the principle of the argument, the number of zeros inside C is given by 
the integer (2z)~'Ac[arg f(z)]. 

It can be shown that the zeros of z++ 2+ 1 lie one in each quadrant. Use the 
above method to show that the zeros in the second and third quadrants have 
\z) <1. 

Prove that 


“ 1 
bare ram = 4n. 
0 4 8 


Carry out the summation numerically, say between —4 and 4, and note how 
much of the sum comes from values near the poles of the contour integration. 
This exercise illustrates a method of summing some infinite series. 


(a) Determine the residues at all the poles of the function 


f(z) = 


where a is a positive real constant. 
(b) By evaluating, in two different ways, the integral J of f(z) along the straight 
line joining —o — ia/2 and +co — ia/2, show that 


TN COt mz 
a+?” 


¥ 1 ncoth za 1 
rr e+n 2a 2a 


(c) Deduce the value of 77 n7?. 


By considering the integral of 


: 2 
sin az 1 T 
— : - a<-cr, 

OZ sin 1z 2 


around a circle of large radius, prove that 


oe) 


(1)! sin? ma il 
De (ma? 


m=1 


Use the Bromwich inversion, and contours similar to that shown in figure 25.7(a), 
to find the functions of which the following are the Laplace transforms: 


(a) s(° +h); 
(b) n!(s—a)“"*), with na positive integer and s > a; 
(c) a(s*— a’), with s > |al. 


Compare your answers with those given in a table of standard Laplace transforms. 
Find the function f(t) whose Laplace transform is 


5 eet Soak 
i 


A function f(t) has the Laplace transform 


1 sti 
F(s) = = In | —~ 
(s) 2i a() ; 
the complex logarithm being defined by a finite branch cut running along the 
imaginary axis from —i to i. 


(a) Convince yourself that, for t > 0, f(t) can be expressed as a closed contour 
integral that encloses only the branch cut. 
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25.15 


25.16 


25.17 


25.18 


(b) Calculate F(s) on either side of the branch cut, evaluate the integral and 
hence determine f(t). 

(c) Confirm that the derivative with respect to s of the Laplace transform 
integral of your answer is the same as that given by dF /ds. 


Use the contour in figure 25.7(c) to show that the function with Laplace transform 
s—/? is (nx) /?. 

[For an integrand of the form r exp(—rx) change variable to t = r'/?.] 
Transverse vibrations of angular frequency @ on a string stretched with constant 
tension T are described by u(x,t) = y(x) e7', where 


1/2 


Here, m(x) = mof (x) is the mass per unit length of the string and, in the general 
case, is a function of x. Find the first-order W.K.B. solution for y(x). 

Due to imperfections in its manufacturing process, a particular string has 
a small periodic variation in its linear density of the form m(x) = mo[1 + 
esin(27x/L)], where e < 1. A progressive wave (i.e. one in which no energy is 
lost) travels in the positive x-direction along the string. Show that its amplitude 
fluctuates by the of its value Ap at x = 0 and that, to first order in e, the phase 


of the wave is 
ewL /m . 4x 
— sin* — 
2n T L 


ahead of what it would be if the string were uniform, with m(x) = mp. 


The equation 
a 1 1 
» { (++ *) y=0, 


dz? 2 4 


sometimes called the Weber—Hermite equation, has solutions known as parabolic 
cylinder functions. Find, to within (possibly complex) multiplicative constants, 
the two W.K.B. solutions of this equation that are valid for large |z|. In each case, 
determine the leading term and show that the multiplicative correction factor is 
of the form 1 + O(v?/z?). 

Identify the Stokes and anti-Stokes lines for the equation. On which of the 
Stokes lines is the W.K.B. solution that tends to zero for z large, real and negative, 
the dominant solution? 

A W.K.B. solution of Bessel’s equation of order zero, 

Py ldy . 

Pore (*) 
valid for large |z| and —x/2 < argz < 3m/2, is y(z) = Az~'/*e”. Obtain an 
improvement on this by finding a multiplier of y(z) in the form of an asymptotic 
expansion in inverse powers of z as follows. 


(a) Substitute for y(z) in (*) and show that the equation is satisfied to O(z~*/). 

(b) Now replace the constant A by A(z) and find the equation that must be 
satisfied by A(z). Look for a solution of the form A(z) = 2° 79 anz~", 
where dp = 1. Show that ¢ = 0 is the only acceptable solution to the indicial 
equation and obtain a recurrence relation for the a,. 

(c) To within a (complex) constant, the expression y(z) = A(z)z~!/2e" is the 
asymptotic expansion of the Hankel function H(z). Show that it is a 
divergent expansion for all values of z and estimate, in terms of z, the value 
of N such that )7™_, a,z~"-"/e" gives the best estimate of H\'(z). 
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25.19 


25.20 


25.21 


25.22 


The function h(z) of the complex variable z is defined by the integral 


100 
h(z) = / exp(t? — 2zt) dt. 
—100 

(a) Make a change of integration variable, t = iu, and evaluate h(z) using a 
standard integral. Is your answer valid for all finite z? 

(b) Evaluate the integral using the method of steepest descents, considering in 
particular the cases (i) z is real and positive, (ii) z is real and negative and 
(iii) z is purely imaginary and equal to if, where f is real. In each case sketch 
the corresponding contour in the complex t-plane. 

(c) Evaluate the integral for the same three cases as specified in part (b) using 
the method of stationary phases. To determine an appropriate contour that 
passes through a saddle point t = to, write t = to + (u+ iv) and apply the 
criterion for determining a level line. Sketch the relevant contour in each 
case, indicating what freedom there is to distort it. 


Comment on the accuracy of the results obtained using the approximate methods 
adopted in (b) and (c). 
Use the method of steepest descents to show that an approximate value for the 
integral 

F(z) = / exp[iz(4t° +t)] dt, 


where z is real and positive, is 
1/2 
(=) expip2) cosipe ~ te, 
where B = 4/(5,/2). 


The stationary phase approximation to an integral of the form 


b 
Fv) = / g(e"dt, vf > 1, 


where f(t) is a real function of t and g(t) is a slowly varying function (when 
compared with the argument of the exponential), can be written as 


an\ SX glts) m " 
F(v) ~ (=) Do Seer {i [sted + FZ sen(vsten) |} 


where the ¢, are the N stationary points of f(t) that lie ina<t)<t2<-+--< 
ty <b, An =|f'"(tn)|, and sgn(x) is the sign of x. 

Use this result to find an approximation, valid for large positive values of v, 
to the integral 


ee a 
Fe 2ye ie Te cos (2 — 320° = 122%1)y] de 
where z is a real positive parameter. 

The Bessel function J,(z) is given for |argz| < 50 by the integral around a 
contour C of the function 


1 Zz 1 
ne ol Oa) = caer: 
g(z) oat exp 5 (« *) : 


The contour starts and ends along the negative real t-axis and encircles the origin 
in the positive sense. It can be considered to be made up of two contours. One 
of them, C2, starts at tf = —oo, runs through the third quadrant to the point 
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25.23 


25.1 


25.3 


25.5 


25.7 


25.9 


25.11 


25.13 


25.15 


t = —i and then approaches the origin in the fourth quadrant in a curve that 
is ultimately antiparallel to the positive real axis. The other contour, Ci, is the 
mirror image of this in the real axis; it is confined to the upper half-plane, passes 
through t =i and is antiparallel to the real t-axis at both of its extremities. The 
contribution to J,(z) from the curve C, is +H‘), the function H being known 
as a Hankel function. 

Using the method of steepest descents, establish the leading term in an asymp- 
totic expansion for H“) for z real, large and positive. Deduce, without detailed 
calculation, the corresponding result for H. Hence establish the asymptotic 
form of J,(z) for the same range of z. 

Use the method of steepest descents to find an asymptotic approximation, valid 
for z large, real and positive, to the function defined by 


F,(z) = iE exp(—iz sint + ivt) dt, 


where v is real and non-negative and C is a contour that starts at t = —1 + ico 
and ends at t = —ioo. 


25.10 Hints and answers 


Apply Kirchhoff’s laws to three independent loops, say ADBA, ADEA and 
DBED. Eliminate other currents from the equations to obtain Ip = MpCVo[ (a — 
w* — 2imao)/(we + w)], where w5 = (LC)~!; |Ir| = @oCVo; the phase of Ip is 
tan“ [ (—2«@0»)/(o} — w) |. 

Set ccothu,; = —d, ccothu, = +d, |ccosech u| = a and note that the capacitance 
is proportional to (uz — u,)71. 

€ =constant, ellipses x7(a+1)~*+y?(a—1)~* = c?/(4a’); = constant, hyperbolae 
x?(cosa)~* — y?(sina)~? = c*. The curves are the cuts —c < x <c, y = 0 and 
|x| = c, y =0. The curves for 7 = 27 are the same as those for n = 0. 

(a) For a quarter-circular contour enclosing the first quadrant, the change in the 
argument of the function is 0 + 8(z/2) + 0 (since y® + 5 = 0 has no real roots); 
(b) one negative real zero; a conjugate pair in the second and third quadrants, 


—3,-1+i. 
Evaluate 
Tm cot mz 
dz 
| (3 +2) (+2) 
around a large circle centred on the origin; residue at z = —1/2 is 0; residue at 
z =—1/4 is 4a cot(—7/4). 
The behaviour of the integrand for large |z| is |z|~? exp [(2«—7)|z| ]. The residue 
at z = +m, for each integer m, is sin?(ma)(—1)"/(ma)?. The contour contributes 
nothing. 
Required summation = [total sum — (m= 0 term) ]/2. 
Note that f(s) has no pole at s = 0. For t < 0 close the Bromwich contour in the 
right half-plane, and for t > 1 in the left half-plane. For 0 < t < 1 the integrand 
has to be split into separate terms containing e~* and s— 1 and the completions 
made in the right and left half-planes, respectively. The last of these completed 
contours now contains a second-order pole at s = 0. f(t) =1—t for0<t< 1, 
but is 0 otherwise. 
Jp and f, tend to 0 as Rw and p > 0. Put s = rexpin and s =rexp(—iz) on 
the two sides of the cut and use f° exp(—t?x) dt = 4(x/x)!/?. There are no poles 
inside the contour. 
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25.17 


25.19 


25.21 


25.23 


Use the binomial theorem to expand, in inverse powers of z, both the square 
root in the exponent and the fourth root in the multiplier, working to O(z~’). 
The leading terms are y(z) = Ce~*"/4z" and y2(z) = De*/4z-"+), Stokes lines: 
argz = 0,2/2,2, 32/2; anti-Stokes lines: argz = (2n + 1)z/4 for n = 0,1,2,3. yy 
is dominant on argz = 2/2 or 37/2. 

(a) ifme~’, valid for all z, including i,/x exp(?) in case (iii). 

(b) The same values as in (a). The (only) saddle point, at to = z, is traversed in 
the direction 0 = +57 in all cases, though the path in the complex t-plane varies 
with each case. 

(c) The same values as in (a). The level lines are v = +u. In cases (i) and (ii) the 
contour turns through a right angle at the saddle point. 

All three methods give exact answers in this case of a quadratic exponent. 
Saddle points at t; = —z and tz = 2z with f/ = —18z and f¥ = 182. 
Approximation is 


( 1 se ae —4n)  cos(20vz3 — jn) 


9zv 1422 14422 


Saddle point at t) = cos!(v/z) is traversed in the direction 9 = —1n. F,(z) © 
(2n/z)'/? exp [i(z — $v — $x)]. 
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Tensors 


It may seem obvious that the quantitative description of physical processes cannot 
depend on the coordinate system in which they are represented. However, we may 
turn this argument around: since physical results must indeed be independent of 
the choice of coordinate system, what does this imply about the nature of the 
quantities involved in the description of physical processes? The study of these 
implications and of the classification of physical quantities by means of them 
forms the content of the present chapter. 

Although the concepts presented here may be applied, with little modifi- 
cation, to more abstract spaces (most notably the four-dimensional space-time of 
special or general relativity), we shall restrict our attention to our familiar three- 
dimensional Euclidean space. This removes the need to discuss the properties of 
differentiable manifolds and their tangent and dual spaces. The reader who is 
interested in these more technical aspects of tensor calculus in general spaces, 
and in particular their application to general relativity, should consult one of the 
many excellent textbooks on the subject $ 

Before the presentation of the main development of the subject, we begin by 
introducing the summation convention, which will prove very useful in writing 
tensor equations in a more compact form. We then review the effects of a change 
of basis in a vector space; such spaces were discussed in chapter 8. This is 
followed by an investigation of the rotation of Cartesian coordinate systems, and 
finally we broaden our discussion to include more general coordinate systems and 
transformations. 


S For example, R. D'Inverno, Introducing Einstein’s Relativity (Oxford: Oxford University Press, 
1992); J. Foster and J. D. Nightingale, A Short Course in General Relativity (New York: Springer, 
2006); B. F. Schutz, A First Course in General Relativity (Cambridge; Cambridge University Press 
1985). 
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26.1 Some notation 


Before proceeding further, we introduce the summation convention for subscripts, 
since its use looms large in the work of this chapter. The convention is that 
any lower-case alphabetic subscript that appears exactly twice in any term of an 
expression is understood to be summed over all the values that a subscript in 
that position can take (unless the contrary is specifically stated). The subscripted 
quantities may appear in the numerator and/or the denominator of a term in an 
expression. This naturally implies that any such pair of repeated subscripts must 
occur only in subscript positions that have the same range of values. Sometimes 
the ranges of values have to be specified but usually they are apparent from the 
context. 
The following simple examples illustrate what is meant (in the three-dimensional 
case): 
(i) a;x; stands for a,x, + a2Xx2 + 43X33 
(ii) ajjbjx, stands for aj bi, + aigbo + ai3b3x5 
(iii) ajjbj.c, stands for ya es aij jeCk 3 
OD; ov ov ov 
(iv) — stands for aes eee a: 
OX; 6x1 0x2 8X3 


oo 0 ed Oo 
Vv stands for —> + =. 
(Vv) OXjOX; Ox? 0x5 OXF 


Subscripts that are summed over are called dummy subscripts and the others 
free subscripts. It is worth remarking that when introducing a dummy subscript 
into an expression, care should be taken not to use one that is already present, 
either as a free or as a dummy subscript. For example, ajjbj,cx; cannot, and must 
not, be replaced by ajjbjjcji or by ajbicxi, but could be replaced by dimbmkcki 
or by dimbmnCu. Naturally, free subscripts must not be changed at all unless the 
working calls for it. 

Furthermore, as we have done throughout this book, we will make frequent 
use of the Kronecker delta 6;;, which is defined by 


1 ifi=j, 
Oj = 
0 otherwise. 
When the summation convention has been adopted, the main use of 6,; is to 
replace one subscript by another in certain expressions. Examples might include 


bjdiz = bi, 
and 
G9 jk = AijOkj = Aik- (26.1) 
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In the second of these the dummy index shared by both terms on the left-hand 
side (namely j) has been replaced by the free index carried by the Kronecker delta 
(namely k), and the delta symbol has disappeared. In matrix language, (26.1) can 
be written as Al = A, where A is the matrix with elements aj; and | is the unit 
matrix having the same dimensions as A. 

In some expressions we may use the Kronecker delta to replace indices in a 
number of different ways, e.g. 


aij jk Oki = ay jb ji or a. jb jk, 


where the two expressions on the RHS are totally equivalent to one another. 


26.2 Change of basis 


In chapter 8 some attention was given to the subject of changing the basis set (or 
coordinate system) in a vector space and it was shown that, under such a change, 
different types of quantity behave in different ways. These results are given in 
section 8.15, but are summarised below for convenience, using the summation 
convention. Although throughout this section we will remind the reader that we 
are using this convention, it will simply be assumed in the remainder of the 
chapter. 

If we introduce a set of basis vectors e;, e2, e3 into our familiar three-dimensional 
(vector) space, then we can describe any vector x in terms of its components 
X1,X2,x3 with respect to this basis: 


X = Xe] + X2€2 + X3€3 = Xjej, 


where we have used the summation convention to write the sum in a more 
compact form. If we now introduce a new basis e/,,e5, e; related to the old one by 


e; = Sijei (sum over i), (26.2) 


where the coefficient Sj; is the ith component of the vector ej with respect to the 
unprimed basis, then we may write x with respect to the new basis as 


x= xe) +x5e,+x3e5 = xe; (sum over i). 


If we denote the matrix with elements S;; by S, then the components x; and x; 
in the two bases are related by 


x, = (S7))iix; (sum over j), 


where, using the summation convention, there is an implicit sum over j from 
j =1to j =3. In the special case where the transformation is a rotation of the 
coordinate axes, the transformation matrix S is orthogonal and we have 


x, =(8");x; =Syxj (sum over j). (26.3) 
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Scalars behave differently under transformations, however, since they remain 
unchanged. For example, the value of the scalar product of two vectors x-y 
(which is just a number) is unaffected by the transformation from the unprimed 
to the primed basis. Different again is the behaviour of linear operators. If a 
linear operator A is represented by some matrix A in a given coordinate system 
then in the new (primed) coordinate system it is represented by a new matrix, 
A’=S"'AS. 

In this chapter we develop a general formulation to describe and classify these 
different types of behaviour under a change of basis (or coordinate transfor- 
mation). In the development, the generic name tensor is introduced, and certain 
scalars, vectors and linear operators are described respectively as tensors of ze- 
roth, first and second order (the order — or rank — corresponds to the number of 
subscripts needed to specify a particular element of the tensor). Tensors of third 
and fourth order will also occupy some of our attention. 


26.3 Cartesian tensors 


We begin our discussion of tensors by considering a particular class of coordinate 
transformation — namely rotations — and we shall confine our attention strictly 
to the rotation of Cartesian coordinate systems. Our object is to study the prop- 
erties of various types of mathematical quantities, and their associated physical 
interpretations, when they are described in terms of Cartesian coordinates and 
the axes of the coordinate system are rigidly rotated from a basis e;,e2,e3 (lying 
along the Ox;, Ox, and Ox3 axes) to a new one e},e},e3 (lying along the Ox', 
Ox, and Ox‘ axes). 

Since we shall be more interested in how the components of a vector or linear 
operator are changed by a rotation of the axes than in the relationship between 
the two sets of basis vectors e; and e;, let us define the transformation matrix L 
as the inverse of the matrix S in (26.2). Thus, from (26.2), the components of a 
position vector x, in the old and new bases respectively, are related by 


x} = Lijx;. (26.4) 


Because we are considering only rigid rotations of the coordinate axes, the 
transformation matrix L will be orthogonal, i. such that L7' = L™. Therefore 
the inverse transformation is given by 


xi = Lyix’. (26.5) 


The orthogonality of L also implies relations among the elements of L that 
express the fact that LL’ = L™'L =|. In subscript notation they are given by 


Li Lijx => Oij and Lyi Lj = Oij. (26.6) 


Furthermore, in terms of the basis vectors of the primed and unprimed Cartesian 
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X24 


x1 


Figure 26.1 Rotation of Cartesian axes by an angle 0 about the x3-axis. The 
three angles marked @ and the parallels (broken lines) to the primed axes 
show how the first two equations of (26.7) are constructed. 


coordinate systems, the transformation matrix is given by 
Lij = e; < ej. 
We note that the product of two rotations is also a rotation. For example, 


suppose that x} = Lijx; and x = Mjj;x;; then the composite rotation is described 
by 


xf = Mix), = MijLjxxk = (MLixxx, 


corresponding to the matrix ML. 


> Find the transformation matrix L corresponding to a rotation of the coordinate axes 
through an angle 0 about the e3-axis (or x3-axis), as shown in figure 26.1. 


Taking x as a position vector — the most obvious choice — we see from the figure that 
the components of x with respect to the new (primed) basis are given in terms of the 
components in the old (unprimed) basis by 


x, = x1 cos 0 + x» sin 0, 
xX) = —x1 sind + x2 cos 0, (26.7) 
X35 = X3. 

The (orthogonal) transformation matrix is thus 


cosO sind 0 
es —sin0 cos@d 0 
0 0 at 


The inverse equations are 
x1 = x}; cos 6 — x4 sin 0, 
x sin 6 + x} cos 0, (26.8) 


a 
X3 = X3, 


X2 
in line with (26.5). < 
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26.4 First- and zero-order Cartesian tensors 


Using the above example as a guide, we may consider any set of three quantities 
vj, which are directly or indirectly functions of the coordinates x; and possibly 
involve some constants, and ask how their values are changed by any rotation of 
the Cartesian axes. The specific question to be answered is whether the specific 
forms v; in the new variables can be obtained from the old ones v; using (26.4), 


v; _ Lijv;- (26.9) 


If so, the v; are said to form the components of a vector or first-order Cartesian 
tensor v. By definition, the position coordinates are themselves the components 
of such a tensor.The first-order tensor v does not change under rotation of the 
coordinate axes; nevertheless, since the basis set does change, from ej, e2, e3 to 
e, e5, V;, the components of v must also change. The changes must be such that 


Vv =vje; = v;e; (26.10) 


is unchanged. 
Since the transformation (26.9) is orthogonal, the components of any such 
first-order Cartesian tensor also obey a relation that is the inverse of (26.9), 


UB= Liv. (26.11) 


We now consider explicit examples. In order to keep the equations to reasonable 
proportions, the examples will be restricted to the x;x2-plane, ie. there are 
no components in the x3-direction. Three-dimensional cases are no different in 
principle — but much longer to write out. 


> Which of the following pairs (v;,v2) form the components of a first-order Cartesian tensor 
in two dimensions? : 


(i) (X2,—x1), (ii) (2,1), (aii) (X7, 9). 


We shall consider the rotation discussed in the previous example, and to save space we 
denote cos 0 by ¢ and sin by s. 


(i) Here v; = x2 and v. = —Xx, referred to the old axes. In terms of the new coordinates 
they will be v} = x, and v, = —x}, ie. 
, rg 
vo, = x = —SX1 + CX? 
} a, (26.12) 
vy = —X) = —CX1 — 8X2. 


Now if we start again and evaluate v; and v} as given by (26.9) we find that 


, 5, —. 
vy = Lyvy + Liv, = cx2 + s(—x1) 


: 26.13 
vy = Lov; + Lav. = —s(X2) + c(—X1). ( ) 


The expressions for v, and v5 in (26.12) and (26.13) are the same whatever the values 
of @ (i.e. for all rotations) and thus by definition (26.9) the pair (x2,—Xx,) is a first-order 
Cartesian tensor. 
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(ii) Here v; = x2 and v2 = x. Following the same procedure, 


, / 

Uy = Xy = —SX1 + CX 
, / 

Vy = Xy = CX, + 8X2. 


But, by (26.9), for a Cartesian tensor we must have 
v} = cv, + sv. = CX? + 8x1 
vy = (—s)v1 + evr = —sx2 + x1. 


These two sets of expressions do not agree and thus the pair (x2,x1) is not a first-order 
Cartesian tensor. 

(iii) v) = x7 and v. = x3. As in (ii) above, considering the first component alone is 
sufficient to show that this pair is also not a first-order tensor. Evaluating v' directly gives 


, , 


2 
v, = x0 = ext + 2esxix2 + 5°x5, 
whilst (26.9) requires that 
v, = cv, + svy = ex, +: 8x3, 


which is quite different. < 


There are many physical examples of first-order tensors (i.e. vectors) that will be 
familiar to the reader. As a straightforward one, we may take the set of Cartesian 
components of the momentum of a particle of mass m, (mx1,mxX2,mx3). This set 
transforms in all essentials as (x1,x2,x3), since the other operations involved, 
multiplication by a number and differentiation with respect to time, are quite 
unaffected by any orthogonal transformation of the axes. Similarly, acceleration 
and force are represented by the components of first-order tensors. 

Other more complicated vectors involving the position coordinates more than 
once, such as the angular momentum of a particle of mass m, namely J = 
X X p = m(x x x), are also first-order tensors. That this is so is less obvious in 
component form than for the earlier examples, but may be verified by writing 
out the components of J explicitly or by appealing to the quotient law to be 
discussed in section 26.7 and using the Cartesian tensor e;;, from section 26.8. 

Having considered the effects of rotations on vector-like sets of quantities we 
may consider quantities that are unchanged by a rotation of axes. In our previous 
nomenclature these have been called scalars but we may also describe them as 
tensors of zero order. They contain only one element (formally, the number of 
subscripts needed to identify a particular element is zero); the most obvious non- 
trivial example associated with a rotation of axes is the square of the distance of 
a point from the origin, r? = x7 + x5 + x3. In the new coordinate system it will 
have the form r? = x)? +x)” +x’, which for any rotation has the same value as 
xt + x3 + x}. 
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In fact any scalar product of two first-order tensors (vectors) is a zero-order 
tensor (scalar), as might be expected since it can be written in a coordinate-free 
way as u-vV. 


>» By considering the components of the vectors u and vy with respect to two Cartesian 
coordinate systems (related by a rotation), show that the scalar product w- v is invariant 
under rotation. 


In the original (unprimed) system the scalar product is given in terms of components by 
ujv; (summed over i), and in the rotated (primed) system by 


on —_ a — 
up, = Lyuj Livy = Lip Ligure = 0 jeujre = ujvj, 


where we have used the orthogonality relation (26.6). Since the resulting expression in the 
rotated system is the same as that in the original system, the scalar product is indeed 
invariant under rotations. < 


The above result leads directly to the identification of many physically im- 
portant quantities as zero-order tensors. Perhaps the most immediate of these is 
energy, either as potential energy or as an energy density (e.g. F- dr, eE- dr, D- E, 
B-H, “-B), but others, such as the angle between two directed quantities, are 
important. 

As mentioned in the first paragraph of this chapter, in most analyses of physical 
situations it is a scalar quantity (such as energy) that is to be determined. Such 
quantities are invariant under a rotation of axes and so it is possible to work with 
the most convenient set of axes and still have confidence in the results. 

Complementing the way in which a zero-order tensor was obtained from two 
first-order tensors, so a first-order tensor can be obtained from a zero-order tensor 
(i.e. a scalar). We show this by taking a specific example, that of the electric field 


E = —V#¢; this is derived from a scalar, the electrostatic potential ¢, and has 
components 
0 
E= mS (26.14) 
OXi 


Clearly, E is a first-order tensor, but we may prove this more formally by 
considering the behaviour of its components (26.14) under a rotation of the 
coordinate axes, since the components of the electric field E/ are then given by 


! at . 
z= ( %0) tO ries (26.15) 


' OX; 


where (26.5) has been used to evaluate 0xj;/0x;. Now (26.15) is in the form 
(26.9), thus confirming that the components of the electric field do behave as the 
components of a first-order tensor. 
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>If v; are the components of a first-order tensor, show that V - v = 0v;/0x; is a zero-order 


tensor. 


In the rotated coordinate system V - v is given by 
A f Ap’ A A 5 
Ov; ov, — 0x; 6 Ov, 
—) = = 1 —(Liv,) = LijLx—, 
(=) Ox; Ox) ae wk) eae Ox; 


since the elements Lj; are not functions of position. Using the orthogonality relation (26.6) 
we then find 


Hence 0v;/6x; is invariant under rotation of the axes and is thus a zero-order tensor; this 
was to be expected since it can be written in a coordinate-free way as V-v. 4 


26.5 Second- and higher-order Cartesian tensors 


Following on from scalars with no subscripts and vectors with one subscript, 
we turn to sets of quantities that require two subscripts to identify a particular 
element of the set. Let these quantities by denoted by T;;. 

Taking (26.9) as a guide we define a second-order Cartesian tensor as follows: 
the Tj; form the components of such a tensor if, under the same conditions as 
for (26.9), 


T;j = Li Lj Tk (26.16) 
and 
Ti = LyiLaj Tp. (26.17) 


At the same time we may define a Cartesian tensor of general order as follows. 
The set of expressions T;;..., form the components of a Cartesian tensor if, for all 
rotations of the axes of coordinates given by (26.4) and (26.5), subject to (26.6), 
the expressions using the new coordinates, Tj, are given by 


Tjek = LipLjq are Lp Tyq.-r (26.18) 


and 
Tij.k = LpiLgj +> Lik rae (26.19) 


It is apparent that in three dimensions, an Nth-order Cartesian tensor has 3% 
components. 

Since a second-order tensor has two subscripts, it is natural to display its 
components in matrix form. The notation [T;;] is used, as well as T, to denote 
the matrix having Tj; as the element in the ith row and jth column 

We may think of a second-order tensor T as a geometrical entity in a similar 
way to that in which we viewed linear operators (which transform one vector into 


S We can also denote the column matrix containing the elements v; of a vector by [v;]. 
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another, without reference to any coordinate system) and consider the matrix 
containing its components as a representation of the tensor with respect to a 
particular coordinate system. Moreover, the matrix T = [Tj], containing the 
components of a second-order tensor, behaves in the same way under orthogonal 
transformations T’ = LTL' as a linear operator. 

However, not all linear operators are second-order tensors. More specifically, 
the two subscripts in a second-order tensor must refer to the same coordinate 
system. In particular, this means that any linear operator that transforms a vector 
into a vector in a different vector space cannot be a second-order tensor. Thus, 
although the elements Lj; of the transformation matrix are written with two 
subscripts, they cannot be the components of a tensor since the two subscripts 
each refer to a different coordinate system. 

As examples of sets of quantities that are readily shown to be second-order 
tensors we consider the following. 

(i) The outer product of two vectors. Let u; and v;, i = 1,2,3, be the components 
of two vectors u and v, and consider the set of quantities Tj; defined by 


Tj => UjVj. (26.20) 


The set Tj; are called the components of the the outer product of u and v. Under 
rotations the components T;; become 


Tj => u, f = Lig uy jv} => LiL jugv} = Li hj Tk, (26.21) 


which shows that they do transform as the components of a second-order tensor. 
Use has been made in (26.21) of the fact that u; and v; are the components of 
first-order tensors. 

The outer product of two vectors is often denoted, without reference to any 
coordinate system, as 


T =u@v. (26.22) 


(This is not to be confused with the vector product of two vectors, which is itself 
a vector and is discussed in chapter 7.) The expression (26.22) gives the basis to 
which the components Tj; of the second-order tensor refer: since u = uje; and 
Vv = vje;, we may write the tensor T as 


T= ue; © vje; = ujvje; @ej = Tij@i @ ej. (26.23) 


Moreover, as for the case of first-order tensors (see equation (26.10)) we note 
that the quantities T;; are the components of the same tensor T, but referred to 
a different coordinate system, i.e. 

S Tyj@; (9) ej => Tie; (39) e’. 


These concepts can be extended to higher-order tensors. 
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(ii) The gradient of a vector. Suppose v; represents the components of a vector; 
let us consider the quantities generated by forming the derivatives of each vj, 
i= 1,2,3, with respect to each xj, j = 1, 2,3, ie. 


These nine quantities form the components of a second-order tensor, as can be 
seen from the fact that 


Ar)! . 
Tj = a = Sa = Ly EL = Li bj Th. 
In coordinate-free language the tensor T may be written as T = Vv and hence 
gives meaning to the concept of the gradient of a vector, a quantity that was not 
discussed in the chapter on vector calculus (chapter 10). 

A test of whether any given set of quantities forms the components of a second- 
order tensor can always be made by direct substitution of the x} in terms of the 
x;, followed by comparison with the right-hand side of (26.16). This procedure is 
extremely laborious, however, and it is almost always better to try to recognise 
the set as being expressible in one of the forms just considered, or to make 
alternative tests based on the quotient law of section 26.7 below. 


> Show that the T; given by 
X30 XX. 
T= (T= ( 3 i) 


Ni? XT 


are the components of a second-order tensor. 


Again we consider a rotation @ about the e3-axis. Carrying out the direct evaluation first 
we obtain, using (26.7), 


1 rd 22 25.2 
Ty, = x4" = 8 X{ — 2scx 1X2 + CX, 
! rt 2 2 2 2 
Ty. = —X1X_ = scx] + (s° — c°)x1x2 — Scx3, 
cad ee eee 2 2 2 2 
Ty, = —X4X5 = scx] + (s° — c°)x1X2 — Scx3, 


Ty = X47 = 7x7 + 2sexy Xo + 57x}. 
Now, evaluating the right-hand side of (26.16), 


Ti, = Wones + cs(—X1X2) + sc(—Xx1X2) + SSX4, 


Tp = c(—s)x3 + ce(—X1X2) + 8(—8)(—X1x2) + sexy, 


Ty = (—s)ex3 + (—s)s(—X1X2) + cc(—X1xX2) + CSX4, 
Thy = (—8)(—8)x3 + (—s)e(—x1x2) + e(—s)(—x1¥9) + cex?, 


After reorganisation, the corresponding expressions are seen to be the same, showing, as 
required, that the T;; are the components of a second-order tensor. 

The same result could be inferred much more easily, however, by noting that the Tj; 
are in fact the components of the outer product of the vector (x2,—x,) with itself. That 
(x2, —x,) is indeed a vector was established by (26.12) and (26.13). < 
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Physical examples involving second-order tensors will be discussed in the later 
sections of this chapter, but we might note here that, for example, magnetic 
susceptibility and electrical conductivity are described by second-order tensors. 


26.6 The algebra of tensors 


Because of the similarity of first- and second-order tensors to column vectors and 
matrices, it would be expected that similar types of algebraic operation can be 
carried out with them and so provide ways of constructing new tensors from old 
ones. In the remainder of this chapter, instead of referring to the Tj; (say) as the 
components of a second-order tensor T, we may sometimes simply refer to Tj; 
as the tensor. It should always be remembered, however, that the Tj; are in fact 
just the components of T in a given coordinate system and that Tj; refers to the 
components of the same tensor T in a different coordinate system. 

The addition and subtraction of tensors follows an obvious definition; namely 
that if Vij... and Wjj..., are (the components of) tensors of the same order, then 
their sum and difference, $j;..., and Djj..., respectively, are given by 


Sij--k = Vij---k Bal Wij---ks 
Dij..-k = Vij..k _ Wij..-ks 


for each set of values i,j,...,k. That Sj... and Djj... are the components of 
tensors follows immediately from the linearity of a rotation of coordinates. 

It is equally straightforward to show that if the Tjj..., are the components of 
a tensor, then so is the set of quantities formed by interchanging the order of (a 
pair of) indices, e.g. Tji...x. 

If Tj... is found to be identical with T;j..., then Tjj..., is said to be symmetric 
with respect to its first two subscripts (or simply ‘symmetric’, for second-order 
tensors). If, however, Tj...4 = —Tjj..., for every element then it is an antisymmetric 
tensor. An arbitrary tensor is neither symmetric nor antisymmetric but can always 
be written as the sum of a symmetric tensor Sj... and an antisymmetric tensor 
Aij..-k is 


Tijk = 3(Tijnk + Tie) + 3(Tipk — Tiie--k) 
= Sij-k + Ajj..-k- 


Of course these properties are valid for any pair of subscripts. 

In (26.20) in the previous section we had an example of a kind of ‘multiplication’ 
of two tensors, thereby producing a tensor of higher order — in that case two 
first-order tensors were multiplied to give a second-order tensor. Inspection of 
(26.21) shows that there is nothing particular about the orders of the tensors 
involved and it follows as a general result that the outer product of an Nth-order 
tensor with an Mth-order tensor will produce an (M + N)th-order tensor. 
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An operation that produces the opposite effect — namely, generates a tensor 
of smaller rather than larger order — is known as contraction and consists of 
making two of the subscripts equal and summing over all values of the equalised 
subscripts. 


> Show that the process of contraction of an Nth-order tensor produces another tensor, of 
order N —2. 


Let Tjj...1...m...k be the components of an Nth-order tensor, then 
Ties adesomeek — LipLijg uae Li ae Lins me Lin TD ng c peigisys 
. — ee 
N factors 


Thus if, for example, we make the two subscripts | and m equal and sum over all values 
of these subscripts, we obtain 


, 
Tyjecteteck = LipLjg ss Lies + Liss ++ Lin Mpg. oreeseeen 
= LipLiq Orso Lay pgs Sen 
= LipLiq: s+ Lin Tyger rns 
een eee! 


(N — 2) factors 


showing that Tij...)...).... are the components of a (different) Cartesian tensor of order 
N-2.< 


For a second-rank tensor, the process of contraction is the same as taking the 
trace of the corresponding matrix. The trace T;; itself is thus a zero-order tensor 
(or scalar) and hence invariant under rotations, as was noted in chapter 8. 

The process of taking the scalar product of two vectors can be recast into tensor 
language as forming the outer product T;; = ujv; of two first-order tensors u and 
vy and then contracting the second-order tensor T so formed, to give Tj; = ujv;, a 
scalar (invariant under a rotation of axes). 

As yet another example of a familiar operation that is a particular case of a 
contraction, we may note that the multiplication of a column vector [uj] by a 
matrix [B,j] to produce another column vector [u)], 


Bijuj = vi, 


can be looked upon as the contraction T;j; of the third-order tensor Tj; formed 
from the outer product of Bj; and ux. 


26.7 The quotient law 


The previous paragraph appears to give a heavy-handed way of describing a 
familiar operation, but it leads us to ask whether it has a converse. To put the 
question in more general terms: if we know that B and C are tensors and also 
that 


Aur iiclg Bitten Oy coca: (26.25) 
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does this imply that the Apg...4...m also form the components of a tensor A? Here 
A, B and C are respectively of Mth, Nth and (M+ N-—2)th order and it should be 
noted that the subscript k that has been contracted may be any of the subscripts 
in A and B independently. 

The quotient law for tensors states that if (26.25) holds in all rotated coordinate 
frames then the Apy...,...m do indeed form the components of a tensor A. To prove 
it for general M and N is no more difficult regarding the ideas involved than to 
show it for specific M and N, but this does involve the introduction of a large 
number of subscript symbols. We will therefore take the case M = N = 2, but 
it will be readily apparent that the principle of the proof holds for general M 
and N. 

We thus start with (say) 


Apr Bix = Cpis (26.26) 


where Bx and C,; are arbitrary second-order tensors. Under a rotation of coor- 
dinates the set A), (tensor or not) transforms into a new set of quantities that 
we will denote by 4’. We thus obtain in succession the following steps, using 
(26.16), (26.17) and (26.6): 


Ay By = Chi (transforming (26.26)), 
= Ly LijCgj (since C is a tensor), 
= Lg hijAq Bil (from (26.26)), 
= Ly hijAg LnjLi Bin, (since B is a tensor), 
= Lyg hint Aq Bin (since LijLnj = Oim)- 


Now &k on the left and n on the right are dummy subscripts and thus we may 
write 


(Axx = Lyg Lit Aq) By = 9. (26.27) 
Since By, and hence Bi,, is an arbitrary tensor, we must have 
Abn = Lpg Lt Aqs 


showing that the A’, are given by the general formula (26.18) and hence that 
the A,, are the components of a second-order tensor. By following an analogous 
argument, the same result (26.27) and deduction could be obtained if (26.26) were 
replaced by 


Apk Bri = Chis 
i.e. the contraction being now with respect to a different pair of indices. 
Use of the quotient law to test whether a given set of quantities is a tensor is 


generally much more convenient than making a direct substitution. A particular 
way in which it is applied is by contracting the given set of quantities, having 
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N subscripts, with an arbitrary Nth-order tensor (i.e. one having independently 
variable components) and determining whether the result is a scalar. 


> Use the quotient law to show that the elements of T, equation (26.24), are the components 
of a second-order tensor. 


The outer product x;x; is a second-order tensor. Contracting this with the Tj; given in 
(26.24) we obtain 


252 2.2 
TijXiX; = XX] Xp XIX 1X2 — Xp X2X2X1 + XX. = 0, 


which is clearly invariant (a zeroth-order tensor). Hence by the quotient theorem Tj; must 
also be a tensor. < 


26.8 The tensors 6;; and €;jx 


In many places throughout this book we have encountered and used the two- 
subscript quantity 6;; defined by 


ee 1 ifi=j, 
y 0 otherwise. 
Let us now also introduce the three-subscript Levi—Civita symbol eijx, the value 
of which is given by 


+1 if i,j,k is an even permutation of 1, 2,3, 
€ijk = 4 —1 if, j,k is an odd permutation of 1, 2, 3, 


0 otherwise. 


We will now show that 6; and jj are respectively the components of a second- 
and a third-order Cartesian tensor. Notice that the coordinates x; do not appear 
explicitly in the components of these tensors, their components consisting entirely 
of 0 and 1. 

In passing, we also note that jj, is totally antisymmetric, i.e. it changes sign 
under the interchange of any pair of subscripts. In fact €;j, or any scalar multiple 
of it, is the only three-subscript quantity with this property. 

Treating 6j; first, the proof that it is a second-order tensor is straightforward 
since if, from (26.16), we consider the equation 


Og = Le Lijdij = Lei Li = x1, 
we see that the transformation of 6;; generates the same expression (a pattern 
of 0’s and 1’s) as does the definition of 4;, in the transformed coordinates. Thus 
6;; transforms according to the appropriate tensor transformation law and is 


therefore a second-order tensor. 
Turning now to e€;;, we have to consider the quantity 


€lmn = LiiLmj Lak €ijk- (26.28) 
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Let us begin, however, by noting that we may use the Levi—Civita symbol to 
write an expression for the determinant of a 3 x 3 matrix A, 


Alem = AjiAmjAnk€ijk> (26.29) 


which may be shown to be equivalent to the Laplace expansion (see chapter 8).$ 
Indeed many of the properties of determinants discussed in chapter 8 can be 
proved very efficiently using this expression (see exercise 26.9). 


> Evaluate the determinant of the matrix 


2 1 -3 
N= | 8 @ 0 
1 —2 1 


Setting / = 1, m = 2 and n = 3 in (26.29) we find 


IA] = €1jA1:A2jA3x 
(2)(4)(1) — (2)(0)(—2) — (1)(3)(1) + (-3)(3)(-2) 
+ (1)(0)(1) — (—3)(4)(1) = 35, 


which may be verified using the Laplace expansion method. < 


We can now show that the €;;, are in fact the components of a third-order tensor. 
Using (26.29) with the general matrix A replaced by the specific transformation 
matrix L, we can rewrite the RHS of (26.28) in terms of |L| 


, 
€lmn = Li Linj Link €ijk = |Lletmn- 


Since L is orthogonal its determinant has the value unity, and so €/,,, = €Imn- 
Thus we see that é’,,,, has exactly the properties of €;;, but with i, j,k replaced by 
I,m,n, ie. it is the same as the expression ej, written using the new coordinates. 
This shows that ej, is a third-order Cartesian tensor. 

In addition to providing a convenient notation for the determinant of a matrix, 
oj; and ej; can be used to write many of the familiar expressions of vector 
algebra and calculus as contracted tensors. For example, provided we are using 
right-handed Cartesian coordinates, the vector product a = b xc has as its 
ith component a; = e€;;xbjcx; this should be contrasted with the outer product 
T =b @¢, which is a second-order tensor having the components Tjj = bjc;. 


S This may be readily extended to an N x N matrix A, ice. 
IAléigin--iy = Ai Ani «+ * Ain iy Cine iy 


where €j,i,...iy equals 1 if iji2---iy is an even permutation of 1,2,...,N and equals —1 if it is an 
odd permutation; otherwise it equals zero. 
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> Write the following as contracted Cartesian tensors: a-b, Vo, V xv, V(V-v), V X(V XY), 
(ax b)-c¢. 


The corresponding (contracted) tensor expressions are readily seen to be as follows: 


a-b= ajb; => dijaib;, 


2, ¢ _. #6 
a Ox;OX; oy Ox;0X;’ 
OVE 
(V X V)i = Eijk 3x)’ 
d ( dv; Ov; 
[V(V -v)];, = oy (=) = OM TO” 


a 2 
0 OUm 0O7Um 

IV x (V x v)]i = ei EkI Eijk Ek 
i ijk Ox; m ax; ij m Ox Ox)" 


(a x b)-¢ = djjcjejaeb) = eiKxiciaeb). <4 


An important relationship between the e- and 6- tensors is expressed by the 
identity 


Eijk€kIm = O10 jm — Oim9 jl- (26.30) 


To establish the validity of this identity between two fourth-order tensors (the 
LHS is a once-contracted sixth-order tensor) we consider the various possible 
cases. 

The RHS of (26.30) has the values 


+l ifi=land j=m4i, (26.31) 
1 ifi=mand j=!F4i, (26.32) 
0 for any other set of subscript values i, j, 1, m. (26.33) 


In each product on the LHS k has the same value in both factors and for a 
non-zero contribution none of i,/, j,m can have the same value as k. Since there 
are only three values, 1, 2 and 3, that any of the subscripts may take, the only 
non-zero possibilities are i=] and j = m or vice versa but not all four subscripts 
equal (since then each e factor is zero, as it would be if i = j or | = m). This 
reproduces (26.33) for the LHS of (26.30) and also the conditions (26.31) and 
(26.32). The values in (26.31) and (26.32) are also reproduced in the LHS of 
(26.30) since 


(i) ifi = 1 and j = m, eije = €mk = €kim and, whether ej; is +1 or —1, the 
product of the two factors is +1; and 

(ii) if i =m and j = 1, eij = Em = —€kim and thus the product €;j,€xim (no 
summation) has the value —1. 


This concludes the establishment of identity (26.30). 
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A useful application of (26.30) is in obtaining alternative expressions for vector 
quantities that arise from the vector product of a vector product. 


> Obtain an alternative expression for V x (V x v). 


As shown in the previous example, V x (V x vy) can be expressed in tensor form as 


[V x (V x v)] tn 
v)]; = & E14) ————_ 
i ijk€klm 8x j0x) 
0m 


0 (=) 070; 
Ox; \ Ox; Ox jOX; 
= (WV -W), - Ve, 
where in the second line we have used the identity (26.30). This result has already 


been mentioned in chapter 10 and the reader is referred there for a discussion of its 
applicability. < 


= (6:10 jm _ 6imOj1) 


By examining the various possibilities, it is straightforward to verify that, more 
generally, 


Oip Oig Oir 
€ijkEpqr = Ojp Ojq Ojr (26.34) 
Okp Okg Okr 


and it is easily seen that (26.30) is a special case of this result. From (26.34) we 
can derive alternative forms of (26.30), for example, 


€;jkEilm = Oj1km — 9 jmOKI- (26.35) 


The pattern of subscripts in these identities is most easily remembered by noting 
that the subscripts on the first 6 on the RHS are those that immediately follow 
(cyclically, if necessary) the common subscript, here i, in each e-term on the LHS; 
the remaining combinations of j,k,/,m as subscripts in the other 6-terms on the 
RHS can then be filled in automatically. 

Contracting (26.35) by setting j =/ (say) we obtain, since dx; = 3 when using 
the summation convention, 


EijkEijm = 35km eae Okm = 26kms 
and by contracting once more, setting k = m, we further find that 


€ijk€ijk = 6. (26.36) 


26.9 Isotropic tensors 


It will have been noticed that, unlike most of the tensors discussed (except for 
scalars), 6;; and e;; have the property that all their components have values 
that are the same whatever rotation of axes is made, i.e. the component values 


944 


26.9 ISOTROPIC TENSORS 


are independent of the transformation Lj;. Specifically, 6), has the value 1 in 
all coordinate frames, whereas for a general second-order tensor T all we know 
is that if Ty; = fi1(x1,x2,x3) then Tj, = fis(x},x5,x3). Tensors with the former 
property are called isotropic (or invariant) tensors. 

It is important to know the most general form that an isotropic tensor can take, 
since the description of the physical properties, e.g. the conductivity, magnetic 
susceptibility or tensile strength, of an isotropic medium (i.e. a medium having 
the same properties whichever way it is orientated) involves an isotropic tensor. 
In the previous section it was shown that 6; and ej; are second- and third-order 
isotropic tensors; we will now show that, to within a scalar multiple, they are the 
only such isotropic tensors. 

Let us begin with isotropic second-order tensors. Suppose Tj; is an isotropic 
tensor; then, by definition, for any rotation of the axes we must have that 


Ti = Tj, = La Lj Tia (26.37) 


for each of the nine components. 

First consider a rotation of the axes by 27/3 about the (1, 1,1) direction; this 
takes Ox;, Ox2, Ox3 into Ox}, Ox}, Ox) respectively. For this rotation L;3 = 1, 
Lo, = 1, L3. = 1 and all other Lj; = 0. This requires that Tj; = Tj, = 133. 
Similarly Tj2 = Tj, = T3. Continuing in this way, we find: 


(a) Ti = Tx: = 133; 
(b) Tix = Tx = T315 
(c) Ty, = Tx = Th. 
Next, consider a rotation of the axes (from their original position) by 2/2 


about the Ox3-axis. In this case Ljz = —1, Lx; = 1, L33 = 1 and all other Lj; = 0. 
Amongst other relationships, we must have from (26.37) that: 


Ty3 =(-1) x 1x Ty; 
T3 =1x 1x Th. 


Hence T;3 = T23 = 0 and therefore, by parts (b) and (c) above, each element Tj; = 
0 except for T,;, T22 and T33, which are all the same. This shows that Tjj = A0j;. 


> Show that Ae; is the only isotropic third-order Cartesian tensor. 


The general line of attack is as above and so only a minimum of explanation will be given. 


Tix = Ty 


ij = LiL jm Len Tinn (in all, there are 27 elements). 


Rotate about the (1, 1,1) direction: this is equivalent to making subscript permutations 
13> 2-3-1. We find 


(a) Ti = T222 = T333, 
(b) Ti12 = Tx3 = T331 ~~ (and two similar sets), 
(c) T123 = Tr31 = T312 (and a set involving odd permutations of 1, 2, 3). 
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Rotate by 2/2 about the Ox3-axis: Liz = —1, Lz = 1, L33 = 1, the other Li; = 0. 


(d) Ti11 = (—1) x (-1) x (-1) x Tx T222, 
Ti12 = (-1) x (-1) x 1 x Tho, 
1x 1x (-1)x Tip, 

(-— 1) x 1x 1x Thy. 


~~ 
ho 
Vo 
of 
2 
ll 


Relations (a) and (d) show that elements with all subscripts the same are zero. Relations 
(e), (f) and (b) show that all elements with repeated subscripts are zero. Relations (g) and 
(c) show that Ty23 = T231 = T312 Tr13 T321 T132. 

In total, Tj; differs from ej by at most a scalar factor, but since ej (and hence Aejjx) 
has already been shown to be an isotropic tensor, Tj, must be the most general third-order 
isotropic Cartesian tensor. < 


Using exactly the same procedures as those employed for 6,; and e;;x, it may be 
shown that the only isotropic first-order tensor is the trivial one with all elements 
zero. 


26.10 Improper rotations and pseudotensors 


So far we have considered rigid rotations of the coordinate axes described by 
an orthogonal matrix L with |L| = +1, (26.4). Strictly speaking such transfor- 
mations are called proper rotations. We now broaden our discussion to include 
transformations that are still described by an orthogonal matrix L but for which 
|L| = —1; these are called improper rotations. 

This kind of transformation can always be considered as an inversion of the 
coordinate axes through the origin represented by the equation 


ec (26.38) 


T 


combined with a proper rotation. The transformation may be looked upon 
alternatively as one that changes an initially right-handed coordinate system into 
a left-handed one; any prior or subsequent proper rotation will not change this 
state of affairs. The most obvious example of a transformation with |L| = —1 is 
the matrix corresponding to (26.38) itself; in this case Lj; = —6jj. 

As we have emphasised in earlier chapters, any real physical vector v may be 
considered as a geometrical object (i.e. an arrow in space), which can be referred 
to independently of any coordinate system and whose direction and magnitude 
cannot be altered merely by describing it in terms of a different coordinate system. 
Thus the components of v transform as v; = Ljjv; under all rotations (proper and 
improper). 

We can define another type of object, however, whose components may also 
be labelled by a single subscript but which transforms as v; = Ljjv; under proper 
rotations and as v; = —Ljjv; (note the minus sign) under improper rotations. In 
this case, the v; are not strictly the components of a true first-order Cartesian 
tensor but instead are said to form the components of a first-order Cartesian 
pseudotensor or pseudovector. 
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Figure 26.2 The behaviour of a vector v and a pseudovector p under a 
reflection through the origin of the coordinate system x1, x2, x3 giving the new 
system x, x5, x}. 


It is important to realise that a pseudovector (as its name suggests) is not a 
geometrical object in the usual sense. In particular, it should not be considered 
as a real physical arrow in space, since its direction is reversed by an improper 
transformation of the coordinate axes (such as an inversion through the origin). 
This is illustrated in figure 26.2, in which the pseudovector p is shown as a broken 
line to indicate that it is not a real physical vector. 

Corresponding to vectors and pseudovectors, zeroth-order objects may be 
divided into scalars and pseudoscalars — the latter being invariant under rotation 
but changing sign on reflection. 

We may also extend the notion of scalars and pseudoscalars, vectors and pseu- 
dovectors, to objects with two or more subscripts. For two subcripts, as defined 
previously, any quantity with components that transform as Tj; = LyxLj Ty un- 
der all rotations (proper and improper) is called a second-order Cartesian tensor. 
If, however, T;; = LiL) Ty under proper rotations but T;; = —Li Lj Th under 
improper ones (which include reflections), then the T;; are the components of 
a second-order Cartesian pseudotensor. In general the components of Cartesian 
pseudotensors of arbitary order transform as 

Tij..-k >= |L| Li Ljm e+ LinTim.-n, (26.39) 
where |L| is the determinant of the transformation matrix. 

For example, from (26.29) we have that 


[Ll eijx = LiL jmLin€imn, 
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but since |L| = +1 we may rewrite this as 
Cijk = |L| LiL jmLkn€imn- 


From this expression, we see that although ¢;,, behaves as a tensor under proper 
rotations, as discussed in section 26.8, it should properly be regarded as a third- 
order Cartesian pseudotensor. 


pIf b; and c, are the components of vectors, show that the quantities a; = €;jxbjcx form 


the components of a pseudovector. 


In a new coordinate system we have 


ec OC 

fo |L| Lit LjmLkn€imnL jpbp Lig lq 
_ |L|Lit€imnOmp Ong bpCq 

= |L|Lit€innDnCn 


= |L|Liai, 


come 
a, = 


from which we see immediately that the quantities a; form the components of a pseu- 
dovector. < 


The above example is worth some further comment. If we denote the vec- 
tors with components b,; and c, by b and ¢ respectively then, as mentioned in 
section 26.8, the quantities a; = €;,bj;c, are the components of the real vector 
a =b xc, provided that we are using a right-handed Cartesian coordinate system. 
However, in a coordinate system that is left-handed the quantitites a; = €/,,bjc; 
are not the components of the physical vector a = b x c, which has, instead, the 
components —di. It is therefore important to note the handedness of a coordinate 
system before attempting to write in component form the vector relation a = bxc 
(which is true without reference to any coordinate system). 

It is worth noting that, although pseudotensors can be useful mathematical 
objects, the description of the real physical world must usually be in terms of 
tensors (ie. scalars, vectors, etc.).) For example, the temperature or density of a 
gas must be a scalar quantity (rather than a pseudoscalar), since its value does 
not change when the coordinate system used to describe it is inverted through 
the origin. Similarly, velocity, magnetic field strength or angular momentum can 
only be described by a vector, and not by a pseudovector. 

At this point, it may be useful to make a brief comment on the distinction 
between active and passive transformations of a physical system, as this difference 
often causes confusion. In this chapter, we are concerned solely with passive trans- 


S In fact the quantum-mechanical description of elementary particles, such as electrons, protons and 
neutrons, requires the introduction of a new kind of mathematical object called a spinor, which is 
not a scalar, vector, or more general tensor. The study of spinors, however, falls beyond the scope 
of this book. 


948 


26.11 DUAL TENSORS 


formations, for which the physical system of interest is left unaltered, and only 
the coordinate system used to describe it is changed. In an active transformation, 
however, the system itself is altered. 

As an example, let us consider a particle of mass m that is located at a position 
x relative to the origin O and hence has velocity x. The angular momentum of 
the particle about O is thus J = m(x x x). If we merely invert the Cartesian 
coordinates used to describe this system through O, neither the magnitude nor 
direction of any these vectors will be changed, since they may be considered 
simply as arrows in space that are independent of the coordinates used to de- 
scribe them. If, however, we perform the analogous active transformation on 
the system, by inverting the position vector of the particle through O, then it 
is clear that the direction of particle’s velocity will also be reversed, since it 
is simply the time derivative of the position vector, but that the direction of 
its angular momentum vector remains unaltered. This suggests that vectors can 
be divided into two categories, as follows: polar vectors (such as position and 
velocity), which reverse direction under an active inversion of the physical sys- 
tem through the origin, and axial vectors (such as angular momentum), which 
remain unchanged. It should be emphasised that at no point in this discus- 
sion have we used the concept of a pseudovector to describe a real physical 
quantity$ 


26.11 Dual tensors 


Although pseudotensors are not themselves appropriate for the description of 
physical phenomena, they are sometimes needed; for example, we may use the 
pseudotensor €;;, to associate with every antisymmetric second-order tensor Ajj 
(in three dimensions) a pseudovector p; given by 


pi = bey A je; (26.40) 


p; is called the dual of A;;. Thus if we denote the antisymmetric tensor A by the 
matrix 


0 Ay —A31 
A=[Aj]=| —-An 0 Ax3 


then the components of its dual pseudovector are (p1, p2, p3) = (A23, A31, Az). 


S The scalar product of a polar vector and an axial vector is a pseudoscalar. It was the experimental 
detection of the dependence of the angular distribution of electrons of (polar vector) momentum 
Pe emitted by polarised nuclei of (axial vector) spin Jn upon the pseudoscalar quantity JN - pe that 
established the existence of the non-conservation of parity in B-decay. 
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> Using (26.40), show that Ajj = €ijx Pk. 


By contracting both sides of (26.40) with ej, we find 
€ikPk = 4€ijk€ktmAtm- 
Using the identity (26.30) then gives 
Eijk Pk = 4(5i10 jm — Simdj1)Aim 
= 5(Aij — Aji) = 3(Ai + Ajj) = Ajj, 


where in the last line we use the fact that Ajj = —Aj;. < 


By a simple extension, we may associate a dual pseudoscalar s with every 
totally antisymmetric third-rank tensor Ajjx, i.e. one that is antisymmetric with 
respect to the interchange of every possible pair of subscripts; s is given by 

1 
s= FCA. (26.41) 

Since Ajj, is a totally antisymmetric three-subscript quantity, we expect it to 

equal some multiple of e;;, (since this is the only such quantity). In fact Ajj, = seijx, 


as can be proved by substituting this expression into (26.41) and using (26.36). 


26.12 Physical applications of tensors 


In this section some physical applications of tensors will be given. First-order 
tensors are familiar as vectors and so we will concentrate on second-order tensors, 
starting with an example taken from mechanics. 

Consider a collection of rigidly connected point particles of which the ath, 
which has mass m” and is positioned at r” with respect to an origin Q, is 
typical. Suppose that the rigid assembly is rotating about an axis through O with 
angular velocity w. 

The angular momentum J about O of the assembly is given by 


5 s (r x p” 
a 


But p® = m@-@ and #@ = w x r™, for any a, and so in subscript form the 
components of J are given by 
Ji = yin Ej jkX j x) 
a 


= («) aC 
= So mM eiex emo xe 
a 


) 
= as m® (545 jm _ 5imdj)XOxe ey QW] 
= = din [() ? $84 — xx] oo = Ino, (26.42) 


where Jj) is a symmetric second-order Cartesian tensor (by the quotient rule, see 
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section 26.7, since J and @ are vectors). The tensor is called the inertia tensor at O 
of the assembly and depends only on the distribution of masses in the assembly 
and not upon the direction or magnitude of w. 

A more realistic situation obtains if a continuous rigid body is considered. In 
this case, m must be replaced everywhere by p(r) dx dy dz and all summations 
by integrations over the volume of the body. Written out in full in Cartesians, 
the inertia tensor for a continuous body would have the form 


{Q? +27)pdV —fxypdv —f{xzpdV 
|= [fj] = —fxypdV f(z? +x*)pdv —fyzpdVv ; 
—fxzpdVv —fyzpdV f(x? +y?)pdVv 


where p = (x,y,z) is the mass distribution and dV stands for dxdydz; the 
integrals are to be taken over the whole body. The diagonal elements of this 
tensor are called the moments of inertia and the off-diagonal elements without the 
minus signs are known as the products of inertia. 


> Show that the kinetic energy of the rotating system is given by T = 311001. 


By an argument parallel to that already made for J, the kinetic energy is given by 
eS sy > m” (¢ ‘ re”) 
a 
= Fo m €.00;X4 €mO1xX 
va 


= $F m5 j15im — 5 jmde1) XQ Xp. OjO1 


a 
2 
Sy [> Ce ee xP x? 9j@1 
a 


1 
= 31 j@jor. 


) 


Alternatively, since J; = Iji@; we may write the kinetic energy of the rotating system as 
T = 4J;0;. < 

The above example shows that the kinetic energy of the rotating body can be 
expressed as a scalar obtained by twice contracting w with the inertia tensor. It 
also shows that the moment of inertia of the body about a line given by the unit 
vector fi is Jf jf (or ATIA in matrix form). 

Since I (= I;;) is a real symmetric second-order tensor, it has associated with it 
three mutually perpendicular directions that are its principal axes and have the 
following properties (proved in chapter 8): 


(i) with each axis is associated a principal moment of inertia 4, u = 1,2,3; 
(ii) when the rotation of the body is about one of these axes, the angular 
velocity and the angular momentum are parallel and given by 


J =Io = j,, 


Le. @ is an eigenvector of I with eigenvalue /,,; 
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(iti) referred to these axes as coordinate axes, the inertia tensor is diagonal 
with diagonal entries 24, A, /3. 


Two further examples of physical quantities represented by second-order tensors 
are magnetic susceptibility and electrical conductivity. In the first case we have 
(in standard notation) 

M; => 4ij Hj, (26.43) 


and in the second case 


Here M is the magnetic moment per unit volume and j the current density 
(current per unit perpendicular area). In both cases we have on the left-hand side 
a vector and on the right-hand side the contraction of a set of quantities with 
another vector. Each set of quantities must therefore form the components of a 
second-order tensor. 

For isotropic media M oc H and j « E, but for anisotropic materials such as 
crystals the susceptibility and conductivity may be different along different crystal 
axes, making y;; and oj; general second-order tensors, although they are usually 
symmetric. 


> The electrical conductivity 6 in a crystal is measured by an observer to have components 


as shown: 
1 a2 0 
[ojl=| V2 3 1 |. (26.45) 
0 ie val 


Show that there is one direction in the crystal along which no current can flow. Does the 
current flow equally easily in the two perpendicular directions? 


The current density in the crystal is given by jj = o;;E;, where o;;, relative to the observer’s 
coordinate system, is given by (26.45). Since [o;j] is a symmetric matrix, it possesses 
three mutually perpendicular eigenvectors (or principal axes) with respect to which the 
conductivity tensor is diagonal, with diagonal entries 21, A, /3, the eigenvalues of [o;)]. 

As discussed in chapter 8, the eigenvalues of [o,j] are given by |o — /l| = 0. Thus we 
require 


from which we find 


(1—4[(3—AU — 4) — 14] -2(1 —4) = 0. 


This simplifies to give 4 = 0,1,4 so that, with respect to its principal axes, the conductivity 
tensor has components aj; given by 


[9%] _ 


oof 
or oO 


0 
0 
0 


Since j; = o/,E’, we see immediately that along one of the principal axes there is no current 


flow and along the two perpendicular directions the current flows are not equal. < 
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We can extend the idea of a second-order tensor that relates two vectors to a 
situation where two physical second-order tensors are related by a fourth-order 
tensor. The most common occurrence of such relationships is in the theory of 
elasticity. This is not the place to give a detailed account of elasticity theory, 
but suffice it to say that the local deformation of an elastic body at any interior 
point P can be described by a second-order symmetric tensor e;; called the strain 


tensor. It is given by 
ent 1 Ou; + Ou; 
wom 2 Ox; OX; ‘ 


where u is the displacement vector describing the strain of a small volume element 


whose unstrained position relative to the origin is x. Similarly we can describe 
the stress in the body at P by the second-order symmetric stress tensor pjj; the 
quantity pj; is the x;-component of the stress vector acting across a plane through 
P whose normal lies in the x;-direction. A generalisation of Hooke’s law then 
relates the stress and strain tensors by 


Pij = Cijkiekl (26.46) 


where cjjxi is a fourth-order Cartesian tensor. 


> Assuming that the most general fourth-order isotropic tensor is 


Cijkt = AOiZOK1 + NOiKO jt + VOi10 jx, (26.47) 


find the form of (26.46) for an isotropic medium having Young’s modulus E and Poisson's 
ratio o. 


For an isotropic medium we must have an isotropic tensor for cjjx;, and so we assume the 
form (26.47). Substituting this into (26.46) yields 
Pij = AOijekx + Ney + VE;ji- 
But e;; is symmetric, and if we write 7 + v = 2y, then this takes the form 
Dij = Ae Oi; + Que, 
in which 4 and yw are known as Lamé constants. It will be noted that if e;; = 0 for i # j 
then the same is true of pj;j, i.e. the principal axes of the stress and strain tensors coincide. 
Now consider a simple tension in the x,-direction, ie. pj; = S but all other pj; = 0. 
Then denoting e,, (summed over k) by 0 we have, in addition to e;; = 0 for i # j, the three 
equations 
S = 204+ 2neu, 
0 = 20 + 2per, 
0 = 20 + 233. 
Adding them gives 
S = 0(34 4+ 2p). 
Substituting for 6 from this into the first of the three, and recalling that Young’s modulus 
is defined by S = Eey,, gives E as 
— WBA+ 2p) 


E= ———. 26.48 
a (26.48) 
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Further, Poisson’s ratio is defined as ¢ = —e2)/e11 (or —e33/e11) and is thus 


1 20 1 A Ee, A 
ef a ef ae — = ; 26.49 
: (x) 5 (x) (=) 3+ 2u 2+ Ww ee 


Solving (26.48) and (26.49) for 2 and wu gives finally 
_ oE 
Pui ~ (Foy — 20) 


E 
Ck ot Atay at 


26.13 Integral theorems for tensors 


In chapter 11, we discussed various integral theorems involving vector and scalar 
fields. Most notably, we considered the divergence theorem, which states that, for 
any vector field a, 


[veady = faraas, (26.50) 
Vv S 


where S is the surface enclosing the volume V and fi is the outward-pointing unit 
normal to S at each point. 
Writing (26.50) in subscript notation, we have 


ye an dV = f avis dS. (26.51) 
Vv OXk Ss 


Although we shall not prove it rigorously, (26.51) can be extended in an obvious 
manner to relate integrals of tensor fields, rather than just vector fields, over 
volumes and surfaces, with the result 


This form of the divergence theorem for general tensors can be very useful in 
vector calculus manipulations. 


>A vector field a satisfies V-a = 0 inside some volume V and a- fi = 0 on the bound- 
ary surface S. By considering the divergence theorem applied to T;j = xj;a;, show that 
JyadV =0. 


Applying the divergence theorem to Tj; = x;a; we find 


oT;; a 
/ —dv = | Mea yp f say ds =0, 
Ox; vy OX; go Es 


since aj; = 0. By expanding the volume integral we obtain 


[ew- seaav+ | sStav 
Vv Vv 


0x; yo 0x; 


= [ djajav 

Vv 

= [ wav =o, 
V 


where in going from the first to the second line we used 0x;/0x; = 6;; and 0a;/0x; =0. < 
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The other integral theorems discussed in chapter 11 can be extended in a 
similar way. For example, written in tensor notation Stokes’ theorem states that, 


for a vector field aj, 
fewstn dS = ¢ ay AX. 


For a general tensor field this has the straightforward extension 
OT im..-k-- A 
fen ivas -¢4 Vinee dx . 
S OX; Cc 


26.14 Non-Cartesian coordinates 


So far we have restricted our attention to the study of tensors when they are 
described in terms of Cartesian coordinates and the axes of coordinates are rigidly 
rotated, sometimes together with an inversion of axes through the origin. In the 
remainder of this chapter we shall extend the concepts discussed in the previous 
sections by considering arbitrary coordinate transformations from one general 
coordinate system to another. Although this generalisation brings with it several 
complications, we shall find that many of the properties of Cartesian tensors 
are still valid for more general tensors. Before considering general coordinate 
transformations, however, we begin by reminding ourselves of some properties of 
general curvilinear coordinates, as discussed in chapter 10. 

The position of an arbitrary point P in space may be expressed in terms of the 
three curvilinear coordinates uj, u2,u3. We saw in chapter 10 that if r(w1, v2, u3) is 
the position vector of the point P then at P there exist two sets of basis vectors 


or 
ee = 5 and e; = Vuj, (26.52) 

Ou; 
where i = 1,2,3. In general, the vectors in each set neither are of unit length nor 
form an orthogonal basis. However, the sets e; and €; are reciprocal systems of 


vectors and so 
ei Eej = Oij. (26.53) 


In the context of general tensor analysis, it is more usual to denote the second 
set of vectors e; in (26.52) by e’, the index being placed as a superscript to 
distinguish it from the (different) vector e;, which is a member of the first set in 
(26.52). Although this positioning of the index may seem odd (not least because 
of the possibility of confusion with powers) it forms part of a slight modification 
to the summation convention that we will adopt for the remainder of this chapter. 
This is as follows: any lower-case alphabetic index that appears exactly twice in 
any term of an expression, once as a subscript and once as a superscript, is to be 
summed over all the values that an index in that position can take (unless the 
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contrary is specifically stated). All other aspects of the summation convention 
remain unchanged. 

With the introduction of superscripts, the reciprocity relation (26.53) should be 
rewritten so that both sides of (26.54) have one subscript and one superscript, Le. 
as 


ee) = 6). (26.54) 


The alternative form of the Kronecker delta is defined in a similar way to 
previously, i.e. it equals unity if i = j and is zero otherwise. 
For similar reasons it is usual to denote the curvilinear coordinates themselves 


by u!,u2,u>, with the index raised, so that 
or j j 
e=s5 and e=Vu'. (26.55) 
ou 


From the first equality we see that we may consider a superscript that appears in 
the denominator of a partial derivative as a subscript. 

Given the two bases e; and e’, we may write a general vector a equally well in 
terms of either basis as follows: 


a=a'le, tae +a°e; = ae; 


a=ae'+me+ae =ae. 


The a! are called the contravariant components of the vector a and the a; 
the covariant components, the position of the index (either as a subscript or 
superscript) serving to distinguish between them. Similarly, we may call the e; the 
covariant basis vectors and the e! the contravariant ones. 


> Show that the contravariant and covariant components of a vector a are given by a' =a-e! 
and a; = a - e; respectively. 


For the contravariant components, we find 
a-e =ale;-e = als; =a', 
where we have used the reciprocity relation (26.54). Similarly, for the covariant components, 


a-e =aje/-e; = a;6] =a;. < 


The reason that the notion of contravariant and covariant components of 
a vector (and the resulting superscript notation) was not introduced earlier is 
that for Cartesian coordinate systems the two sets of basis vectors e; and e! are 
identical and, hence, so are the components of a vector with respect to either 
basis. Thus, for Cartesian coordinates, we may speak simply of the components 
of the vector and there is no need to differentiate between contravariance and 
covariance, or to introduce superscripts to make a distinction between them. 

If we consider the components of higher-order tensors in non-Cartesian co- 
ordinates, there are even more possibilities. As an example, let us consider a 
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second-order tensor T. Using the outer product notation in (26.23), we may write 
T in three different ways: 


T = Te; @ ej = T'je; @ e/ = Tye! @ e!, 


where TY, TT, and Tj; are called the contravariant, mixed and covariant com- 
ponents of T respectively. It is important to remember that these three sets of 
quantities form the components of the same tensor T but refer to different (tensor) 
bases made up from the basis vectors of the coordinate system. Again, if we are 
using Cartesian coordinates then all three sets of components are identical. 

We may generalise the above equation to higher-order tensors. Components 
carrying only superscripts or only subscripts are referred to as the contravariant 
and covariant components respectively; all others are called mixed components. 


26.15 The metric tensor 


Any particular curvilinear coordinate system is completely characterised at each 
point in space by the nine quantities 


gg = epee, (26.56) 


which, as we will show, are the covariant components of a symmetric second-order 
tensor g called the metric tensor. 

Since an infinitesimal vector displacement can be written as dr = dule;, we find 
that the square of the infinitesimal arc length (ds)? can be written in terms of the 
metric tensor as 


(ds) = dr- dr = due; - dule; = gij duidu/. (26.57) 
It may further be shown that the volume element dV is given by 
dV = /gdu' du’ du’, (26.58) 


where g is the determinant of the matrix [ g;;], which has the covariant components 
of the metric tensor as its elements. 

If we compare equations (26.57) and (26.58) with the analogous ones in section 
10.10 then we see that in the special case where the coordinate system is orthogonal 
(so that ej-e; = 0 for i # j) the metric tensor can be written in terms of the 
coordinate-system scale factors h;, i = 1,2,3 as 


_ she isi, 
ae Vs it j. 


Its determinant is then given by g = hih3h3. 
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> Calculate the elements gj; of the metric tensor for cylindrical polar coordinates. Hence 
find the square of the infinitesimal arc length (ds)* and the volume dV for this coordinate 
system. 


As discussed in section 10.9, in cylindrical polar coordinates (u!,u?,u*) = (p,¢,z) and so 
the position vector r of any point P may be written 


r=pcosdi+psing j+ zk. 


From this we obtain the (covariant) basis vectors: 


gates = cosdi+sing j; 
op 
or , : . 
= 35 = —psingdi+pcos¢ j; 
or 
=— =k 26.59 
&3 Oz ( ) 


Thus the components of the metric tensor [gi] = [e; - ej] are found to be 


1 0 0 
G=[g)=[ 0 p° 0 |, (26.60) 
0 0 1 


from which we see that, as expected for an orthogonal coordinate system, the metric tensor 
is diagonal, the diagonal elements being equal to the squares of the scale factors of the 
coordinate system. 

From (26.57), the square of the infinitesimal arc length in this coordinate system is given 
by 


(ds)? = gij du‘ du/ = (dpy + p*(doy + (azy, 
and, using (26.58), the volume element is found to be 
dV = Vg du' dw? du? = p dp do dz. 
These expressions are identical to those derived in section 10.9. < 
We may also express the scalar product of two vectors in terms of the metric 
tensor: 
a-b=ale;- ble; = gija'b/, (26.61) 
where we have used the contravariant components of the two vectors. Similarly, 
using the covariant components, we can write the same scalar product as 
a-b=aze'- bje! = gajb;, (26.62) 


where we have defined the nine quantities g'/ = e'-e/. As we shall show, they form 
the contravariant components of the metric tensor g and are, in general, different 
from the quantities g;j. Finally, we could express the scalar product in terms of 
the contravariant components of one vector and the covariant components of the 
other, 


a-b=aje'- ble; = a;b/6i =a;b', (26.63) 


958 


26.15 THE METRIC TENSOR 


where we have used the reciprocity relation (26.54). Similarly, we could write 
a-b=a'e;- bye! = a'bjo! = a'b;. (26.64) 


By comparing the four alternative expressions (26.61)-(26.64) for the scalar 
product of two vectors we can deduce one of the most useful properties of 
the quantities g;; and g/. Since g;ja'b/ = a'b; holds for any arbitrary vector 
components a, it follows that 


gijb/ = bi, 


which illustrates the fact that the covariant components g;; of the metric tensor 
can be used to lower an index. In other words, it provides a means of obtaining 
the covariant components of a vector from its contravariant components. By a 
similar argument, we have 


lb; = bi, 


so that the contravariant components g‘/ can be used to perform the reverse 
operation of raising an index. 

It is straightforward to show that the contravariant and covariant basis vectors, 
e! and e; respectively, are related in the same way as other vectors, ie. by 


e' = gile; and =e, = gije’. 


We also note that, since e; and e’ are reciprocal systems of vectors in three- 
dimensional space (see chapter 7), we may write 
ej; X & 
ei PERE EIC 
ej - (e; X ex) 


e = 
for the combination of subscripts i, j,k = 1,2,3 and its cyclic permutations. A 
similar expression holds for e; in terms of the e’-basis. Moreover, it may be shown 


that |e; - (e2 x e3)| = JS 


> Show that the matrix [g'/] is the inverse of the matrix [gij]. Hence calculate the con- 
travariant components g'! of the metric tensor in cylindrical polar coordinates. 


Using the index-lowering and index-raising properties of g;; and g!/ on an arbitrary vector 
a, we find 


i ee | ee ee 
o.a =a = gay = gi gia. 


But, since a is arbitrary, we must have 


eign = OL. (26.65) 

Denoting the matrix [g;;] by G and [g‘/] by G, equation (26.65) can be written in matrix 

form as GG = |, where | is the unit matrix. Hence G and G are inverse matrices of each 
other. 
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Thus, by inverting the matrix G in (26.60), we find that the elements g’’ are given in 
cylindrical polar coordinates by 


; 1 0 
G=[e]=| 0 1/p? 0 |.<« 
0 1 


So far we have not considered the components of the metric tensor gi with one 
subscript and one superscript. By analogy with (26.56), these mixed components 
are given by 


Dalat _ sj 
gj =e e; = 0;, 


and so the components of gi are identical to those of 5}. We may therefore 
consider the 5} to be the mixed components of the metric tensor g. 


26.16 General coordinate transformations and tensors 


We now discuss the concept of general transformations from one coordinate 


system, u',u?,u°, to another, ull, uu. We can describe the coordinate transform 


using the three equations 
ri rin 1 2 3 
usu (uuu), 


for i = 1, 2,3, in which the new coordinates u’' can be arbitrary functions of the old 
ones uw! rather than just represent linear orthogonal transformations (rotations) 
of the coordinate axes. We shall assume also that the transformation can be 
inverted, so that we can write the old coordinates in terms of the new ones as 
ul = ui(u!' ul? u”) 
As an example, we may consider the transformation from spherical polar to 
Cartesian coordinates, given by 


x =rsin0cos ¢, 

y =rsin0sin @, 

z=rcos0, 
which is clearly not a linear transformation. 


The two sets of basis vectors in the new coordinate system, w!!, u’?, u 
as in (26.55) by 


‘3 are given 


Al is 
e=— and ei =vu' (26.66) 
ou" 


Considering the first set, we have from the chain rule that 


or ou" or 
ous ous dy” 
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so that the basis vectors in the old and new coordinate systems are related by 


bul’ 
ej = e (26.67) 


dus * 
Now, since we can write any arbitrary vector a in terms of either basis as 
Ou! ; 


a I es ee ae os, 
a=ae,=a’ej; =a’°—e 
i J oul 


i> 
it follows that the contravariant components of a vector must transform as 


: bu! 
ni Y j 
= — a’. 26.68 
ous ( ) 
In fact, we use this relation as the defining property for a set of quantities a! to 
form the contravariant components of a vector. 


> Find an expression analogous to (26.67) relating the basis vectors e' and e" in the two 


coordinate systems. Hence deduce the way in which the covariant components of a vector 
change under a coordinate transformation. 


If we consider the second set of basis vectors in (26.66), e’ = Vu", we have from the chain 
rule that 


dus ut Ou" 
ox du"! dx 


and similarly for éu//dy and éu//dz. So the basis vectors in the old and new coordinate 
systems are related by 


/ ous 
e= Fall if (26.69) 
For any arbitrary vector a, 
ma i ou! ni 
a=ae =aje’ =aj— 
ou’ 
and so the covariant components of a vector must transform as 
dul 
d, = ayn (26.70) 
u 


Analogously to the contravariant case (26.68), we take this result as the defining property 
of the covariant components of a vector. < 


We may compare the transformation laws (26.68) and (26.70) with those for 
a first-order Cartesian tensor under a rigid rotation of axes. Let us consider 
a rotation of Cartesian axes x! through an angle 0 about the 3-axis to a new 
set x", i = 1,2,3, as given by (26.7) and the inverse transformation (26.8). It is 
straightforward to show that 


Oxi Ax" 


es ee 
Ox!’ = 0x4 2 


TENSORS 


where the elements Lj; are given by 


cos0 sind 0 
L= —sin@ cosd 0 
0 0 1 


Thus (26.68) and (26.70) agree with our earlier definition in the special case of a 
rigid rotation of Cartesian axes. 

Following on from (26.68) and (26.70), we proceed in a similar way to de- 
fine general tensors of higher rank. For example, the contravariant, mixed and 
covariant components, respectively, of a second-order tensor must transform as 
follows: 
ou bu 
duk dul : 

; ou" dul 
mixed components, T,=——T" ; 


J uk 6ut 2 


. ij 
contravariant components, T= 


d ; éuk éu! 
covariant components, Tij= -—— Tj). 
ou" ou! 


It is important to remember that these quantities form the components of the 
same tensor T but refer to different tensor bases made up from the basis vectors 
of the different coordinate systems. For example, in terms of the contravariant 
components we may write 

T= Te @e, =T'e,@ e}. 

We can clearly go on to define tensors of higher order, with arbitrary numbers 
of covariant (subscript) and contravariant (superscript) indices, by demanding 
that their components transform as follows: 


qitiee ul! aul du Gut due auf as er 
bak nae : T moo" n de---f* (26.71) 
out du ue 6y!! du’ ou! 


Using the revised summation convention described in section 26.14, the algebra 
of general tensors is completely analogous to that of the Cartesian tensors 
discussed earlier. For example, as with Cartesian coordinates, the Kronecker 
delta is a tensor provided it is written as the mixed tensor 5} since 


where we have used the chain rule to justify the third equality. This also shows 
that 5} is isotropic. As discussed at the end of section 26.15, the 5} can be 
considered as the mixed components of the metric tensor g. 
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> Show that the quantities gj; = e;-e; form the covariant components of a second-order 


tensor. 


In the new (primed) coordinate system we have 


hn i tat 
Bij = &* Ej 


but using (26.67) for the inverse transformation, we have 


, uk 
eS oe 
0 
and similarly for ej. Thus we may write 
,  euk éul ou* 6ul 
= ——— & ee = — — 
Sij éu" éu kel éu" eu Ski 


which shows that the gi; are indeed the covariant components of a second-order tensor 
(the metric tensor g). < 


A similar argument to that used in the above example shows that the quantities 
g/ form the contravariant components of a second-order tensor which transforms 
according to 


- a, fi aati 
nj Ou” du” Ey 


~ Buk dul © 

In the previous section we discussed the use of the components g;; and g'/ in 
the raising and lowering of indices in contravariant and covariant vectors. This 
can be extended to tensors of arbitrary rank. In general, contraction of a tensor 
with g,; will convert the contracted index from being contravariant (superscript) 
to covariant (subscript), i.e. it is lowered. This can be repeated for as many indices 
are required. For example, 


Ti = geT*; = gngaT™. (26.72) 
Similarly contraction with g'/ raises an index, ice. 
TY = g* TJ = g* gl Ty. (26.73) 


That (26.72) and (26.73) are mutually consistent may be shown by using the fact 
that gg, ; = di. 


26.17 Relative tensors 


In section 26.10 we introduced the concept of pseudotensors in the context of the 
rotation (proper or improper) of a set of Cartesian axes. Generalising to arbitrary 
coordinate transformations leads to the notion of a relative tensor. 

For an arbitrary coordinate transformation from one general coordinate system 
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u' to another uw’, we may define the Jacobian of the transformation (see chapter 6) 
as the determinant of the transformation matrix [ou /du4]: this is usually denoted 
by 

oul 


Ou 
Alternatively, we may interchange the primed and unprimed coordinates to 
obtain |du/du'| = 1/J: unfortunately this also is often called the Jacobian of the 
transformation. 

Using the Jacobian J, we define a relative tensor of weight w as one whose 
components transform as follows: 


7 rin aj atk ad e Af 
prinek _ oul du ou™ du* Ou OW abc 


Im--n Out uP ouc aul! ou” oul” de---f 


(26.74) 


Comparing this expression with (26.71), we see that a true (or absolute) general 
tensor may be considered as a relative tensor of weight w = 0. If w = —1, on the 
other hand, the relative tensor is known as a general pseudotensor and if w = 1 
as a tensor density. 

It is worth comparing (26.74) with the definition (26.39) of a Cartesian pseu- 
dotensor. For the latter, we are concerned only with its behaviour under a rotation 
(proper or improper) of Cartesian axes, for which the Jacobian J = +1. Thus, 
general relative tensors of weight w = —1 and w = 1 would both satisfy the 
definition (26.39) of a Cartesian pseudotensor. 


>If the gi; are the covariant components of the metric tensor, show that the determinant g 
of the matrix [gij| is a relative scalar of weight w = 2. 


The components g;; transform as 
, _ duk du! 
81 = ay ayT® 
Defining the matrices U = [éu'/du'/], G = [gij] and G’ = [gi], we may write this expression 
as 


kl- 


G’ =U'GU. 
Taking the determinant of both sides, we obtain 


2 
8, 


ou 


g =|UPg = 


ou’ 
which shows that g is a relative scalar of weight w = 2. < 

From the discussion in section 26.8, it can be seen that ejj, is a covariant 
relative tensor of weight —1. We may also define the contravariant tensor e!/*, 


which is numerically equal to ¢;;, but is a relative tensor of weight +1. 
If two relative tensors have weights w; and w respectively then, from (26.74), 
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the outer product of the two tensors, or any contraction of them, is a relative 
tensor of weight w; + w2. As a special case, we may use ej; and e@* to construct 
pseudovectors from antisymmetric tensors and vice versa, in an analogous way 
to that discussed in section 26.11. 

For example, if the A‘/ are the contravariant components of an antisymmetric 
tensor (w = 0) then 


a il 
Pi = 361k A? 


are the covariant components of a pseudovector (w = —1), since e;;, has weight 
w =—1. Similarly, we may show that 
AU = eliF py. 


26.18 Derivatives of basis vectors and Christoffel symbols 


In Cartesian coordinates, the basis vectors e; are constant and so their derivatives 
with respect to the coordinates vanish. In a general coordinate system, however, 
the basis vectors e; and e! are functions of the coordinates. Therefore, in order 
that we may differentiate general tensors we must consider the derivatives of the 
basis vectors. 

First consider the derivative de;/du/. Since this is itself a vector, it can be 
written as a linear combination of the basis vectors e,, k = 1,2,3. If we introduce 
the symbol I, ; to denote the coefficients in this combination, we have 


ee ee (26.75) 


The coefficient , is the kth component of the vector de;/éu/. Using the reci- 
procity relation e!- e; = Ji, these 27 numbers are given (at each point in space) 
by 


re =ek. Oe) 
uy ous 


(26.76) 


Furthermore, by differentiating the reciprocity relation e! - ej = 6} with respect 
to the coordinates, and using (26.76), it is straightforward to show that the 
derivatives of the contravariant basis vectors are given by 


Ge! 


ous 


—T',e*. (26.77) 


The symbol r, is called a Christoffel symbol (of the second kind), but, despite 
appearances to the contrary, these quantities do not form the components of a 
third-order tensor. It is clear from (26.76) that in Cartesian coordinates r,, =0 
for all values of the indices i, j and k. 
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> Using (26.76), deduce the way in which the quantities Dy transform under a general 


coordinate transformation, and hence show that they do not form the components of a 
third-order tensor. 


In a new coordinate system 


but from (26.69) and (26.67) respectively we have, on reversing primed and unprimed 
variables, 

5, 7k >| 

= ie : and = ow. 

ou" ou" 


Therefore in the new coordinate system the quantities i, are given by 


k 
aw (Ou, ful 
T y= e-—|=—e 
J Ou" ou! \ du" 


du ul * dul de; 
= —e". | ———_e, + — — 
ou" ou’ ou" ou" du! 


e;. 


k ka. tos B 
us Ou, éu™ ou! du", 6e 
= Ge aylout © “" Grayta ©) aye 
Ou" du ou Ou" ul Ou’! Ou 
A1'* 2a A1/* I m 
ou™ Oru odu™ du’ du 
= + I” (26.78) 


dul dullou® Gu" dul Out 

where in the last line we have used (26.76) and the reciprocity relation e” - e) = 6;'. From 
(26.78), because of the presence of the first term on the right-hand side, we conclude 
immediately that the I‘, do not form the components of a third-order tensor. < 

In a given coordinate system, in principle we may calculate the I using 
(26.76). In practice, however, it is often quicker to use an alternative expression, 
which we now derive, for the Christoffel symbol in terms of the metric tensor gj; 
and its derivatives with respect to the coordinates. 

Firstly we note that the Christoffel symbol rm, is symmetric with respect to 
the interchange of its two subscripts i and j. This is easily shown: since 


0e; er or ej 


dus Ousdui — Oul'dus Bu’ 
it follows from (26.75) that Te = TD ek. Taking the scalar product with e’ and 
using the reciprocity relation e, - e! = 6/ gives immediately that 
1 I 
Py =U je 
To obtain an expression for rs, we then use gj; = e;-e; and consider the 


derivative 
gi; 0e; 6e; 


ee et arate gle Mee 
ouk ~~ Guk uk 
| | 
=Wxer- ej +e T per 
| | 
= e8y +1 8a (26.79) 
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where we have used the definition (26.75). By cyclically permuting the free indices 
i, j,k in (26.79), we obtain two further equivalent relations, 


Lie eres ee 26.80 

Bye Stk + Si (26.80) 
and. 7 

O8ki 

Sor = Pen + Piygu (26.81) 


If we now add (26.80) and (26.81) together and subtract (26.79) from the result, 
we find 


O8 jk F Ogi — OB 
dui * dus Ouk 


=T ge tM aga +O ygu + Ug —Tggy —P agi 
= 20 jkr, 
where we have used the symmetry properties of both r,, and gjj. Contracting 


both sides with g”* leads to the required expression for the Christoffel symbol in 
terms of the metric tensor and its derivatives, namely 


ae Deis. 2d 
rm, = 4g ( 4 28k _ “tt (26.82) 


oul Gus — uk 


> Calculate the Christoffel symbols 1, for cylindrical polar coordinates. 


We may use either (26.75) or (26.82) to calculate the I”), for this simple coordinate system. 
In cylindrical polar coordinates (u',u?,u>) = (p,@,z), the basis vectors e; are given by 
(26.59). It is straightforward to show that the only derivatives of these vectors with respect 
to the coordinates that are non-zero are 


0ep 1. 0eg 1. Oey oe 
ag p” dp p®*” a6 e 


Thus, from (26.75), we have immediately that 


1 
I’, =I, = Z and r',, =—p. (26.83) 


Alternatively, using (26.82) and the fact that gi; = 1, go = p’, g33 = 1 and the other 
components are zero, we see that the only three non-zero Christoffel symbols are indeed 
I?,, =I, and I'!,,. These are given by 


1 0gx 1 0 1 
2 SS a fp?) a 
12 21 Qen oul 2p? ap? ) Fe 
1 0g 10 2 
r 
22 eu oul 2 ap? ) Pp, 


which agree with the expressions found directly from (26.75) and given in (26.83 ). < 
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26.19 Covariant differentiation 


For Cartesian tensors we noted that the derivative of a scalar is a (covariant) 
vector. This is also true for general tensors, as may be shown by considering the 
differential of a scalar 
0 : 
db = o du'. 
Since the du’ are the components of a contravariant vector and d¢ is a scalar, 
we have by the quotient law, discussed in section 26.7, that the quantities 0¢/du'! 
must form the components of a covariant vector. As a second example, if the 
contravariant components in Cartesian coordinates of a vector v are v', then the 
quantities dv'/Ax/ form the components of a second-order tensor. 
However, it is straightforward to show that in non-Cartesian coordinates differ- 
entiation of the components of a general tensor, other than a scalar, with respect 
to the coordinates does not in general result in the components of another tensor. 


> Show that, in general coordinates, the quantities dv'/du/ do not form the components of 
a tensor. 


We may show this directly by considering 
re ; , 
éu' 6v" —duk av" 
dus du dul uk 
duk 6 fou" | 
= —— | —0 
ous duk \ ou! 


duk dul" dv! duk Pu", 


Ou Ow uk ' Gul Oukou Coe) 
The presence of the second term on the right-hand side of (26.84) shows that the év'/éx/ 
do not form the components of a second-order tensor. This term arises because the 
‘transformation matrix’ [du"'/du“] changes as the position in space at which it is evaluated 
is changed. This is not true in Cartesian coordinates, for which the second term vanishes 
and dv'/dx/ is a second-order tensor. < 


We may, however, use the Christoffel symbols discussed in the previous section 
to define a new covariant derivative of the components of a tensor that does 
result in the components of another tensor. 

Let us first consider the derivative of a vector v with respect to the coordinates. 
Writing the vector in terms of its contravariant components v = v'e;, we find 

dv vi ; Oe; 


itv 


- -@ 
ous Oud 


(26.85) 


dus’ 
where the second term arises because, in general, the basis vectors e; are not 
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constant (this term vanishes in Cartesian coordinates). Using (26.75) we write 


Since i and k are dummy indices in the last term on the right-hand side, we may 
interchange them to obtain 


OV év' : dv! ee 
bul — ay + YT, ej — (= + a) ej. (26.86) 


The reason for the interchanging the dummy indices, as shown in (26.86), is that 
we may now factor out e;. The quantity in parentheses is called the covariant 
derivative, for which the standard notation is 


avi. 
oes, (26.87) 


the semicolon subscript denoting covariant differentiation. A similar short-hand 
notation also exists for the partial derivatives, a comma being used for these 
instead of a semicolon; for example, dv'/du/ is denoted by vi. In Cartesian 
coordinates all the I”,; are zero, and so the covariant derivative reduces to the 
simple partial derivative dv'/du/. 

Using the short-hand semicolon notation, the derivative of a vector may be 
written in the very compact form 


ov 
7 =U. ej 
ous J 


and, by the quotient rule (section 26.7), it is clear that the vi are the (mixed) 
components of a second-order tensor. This may also be verified directly, using 
the transformation properties of dv'/du/ and T',, given in (26.84) and (26.78) 
respectively. 

In general, we may regard the vj as the mixed components of a second- 
order tensor called the covariant derivative of v and denoted by Vv. In Cartesian 


coordinates, the components of this tensor are just 6v'/dx/. 


> Calculate v'., in cylindrical polar coordinates. 


Contracting (26.87) we obtain 


Now from (26.83) we have 
My = My ale Pp aR Pa =1/p, 
M;; =I"5 +15 +1353 = 0, 
I"; =I", +13 +1333 = 0, 
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and so 
, _ Ov dv? dv? ie 1 
“6p Oo O62 p 
10 6v® av’ 
P 
pope ode oe 


This result is identical to the expression for the divergence of a vector field in cylindrical 
polar coordinates given in section 10.9. This is discussed further in section 26.20. < 


So far we have considered only the covariant derivative of the contravariant 
components v! of a vector. The corresponding result for the covariant components 
v; may be found in a similar way, by considering the derivative of vy = vje’ and 
using (26.77) to obtain 


—T*, vp. (26.88) 


Comparing the expressions (26.87) and (26.88) for the covariant derivative of 
the contravariant and covariant components of a vector respectively, we see that 
there are some similarities and some differences. It may help to remember that 
the index with respect to which the covariant derivative is taken (j in this case), is 
also the last subscript on the Christoffel symbol; the remaining indices can then 
be arranged in only one way without raising or lowering them. It only remains to 
note that for a covariant index (subscript) the Christoffel symbol carries a minus 
sign, whereas for a contravariant index (superscript) the sign is positive. 

Following a similar procedure to that which led to equation (26.87), we may 
obtain expressions for the covariant derivatives of higher-order tensors. 


> By considering the derivative of the second-order tensor T with respect to the coordinate 
u‘, find an expression for the covariant derivative T".,, of its contravariant components. 


Expressing T in terms of its contravariant components, we have 


oT 0 4 
aut ~ Bet Oe) 
oT ;, Oe; ‘s de; 
=> er fi ® ej + Dee ® ej + Te; ® Buk 


Using (26.75), we can rewrite the derivatives of the basis vectors in terms of Christoffel 
symbols to obtain 


oT OT’ +9 a 
ak = Bare Ge + TT ,e: @e; + Te, ® Ve. 


Interchanging the dummy indices i and / in the second term and j and / in the third term 
on the right-hand side, this becomes 


oT 6Ti , ‘ ated 
(= -+PyT! + 47") e; @ ej, 


ouk 
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where the expression in parentheses is the required covariant derivative 


ij OT F ; pe 
P= art i, TY +17". (26.89) 
Using (26.89), the derivative of the tensor T with respect to u* can now be written in terms 
of its contravariant components as 
oT 8, 
as T’ ,e @e;. < 
Results similar to (26.89) may be obtained for the the covariant derivatives of 
the mixed and covariant components of a second-order tensor. Collecting these 


results together, we have 
2 a Feta ad 
T a ae 4 r,T j ale sey : 
i i il Lomi 
T' j-k = T ik + "nT; a MT, 
Tijsk = Tijk —T'e Ty — an Til, 


where we have used the comma notation for partial derivatives. The position of 
the indices in these expressions is very systematic: for each contravariant index 
(superscript) on the LHS we add a term on the RHS containing a Christoffel 
symbol with a plus sign, and for every covariant index (subscript) we add a 
corresponding term with a minus sign. This is extended straightforwardly to 
tensors with an arbitrary number of contravariant and covariant indices. 

We note that the quantities Tee Tg and T;;., are the components of the 
same third-order tensor VT with respect to different tensor bases, i.e. 


VT = TY ej @e,@e =T';,e, @e/ Be = Tij,,e' Be! Bet. 


We conclude this section by considering briefly the covariant derivative of a 
scalar. The covariant derivative differs from the simple partial derivative with 
respect to the coordinates only because the basis vectors of the coordinate 
system change with position in space (hence for Cartesian coordinates there is no 
difference). However, a scalar ¢ does not depend on the basis vectors at all and 
so its covariant derivative must be the same as its partial derivative, i.e. 

a 
ob: = 2 = d.j- (26.90) 


ous 
26.20 Vector operators in tensor form 


In section 10.10 we used vector calculus methods to find expressions for vector 
differential operators, such as grad, div, curl and the Laplacian, in general orthog- 
onal curvilinear coordinates, taking cylindrical and spherical polars as particular 
examples. In this section we use the framework of general tensors that we have 
developed to obtain, in tensor form, expressions for these operators that are valid 
in all coordinate systems, whether orthogonal or not. 
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In order to compare the results obtained here with those given in section 
10.10 for orthogonal coordinates, it is necessary to remember that here we are 
working with the (in general) non-unit basis vectors e; = ér/du' or e! = Vui. 
Thus the components of a vector v = ve; are not the same as the components 6! 
appropriate to the corresponding unit basis é;. In fact, if the scale factors of the 
coordinate system are h,, i = 1,2,3, then v' = 6'/h; (no summation over i). 

As mentioned in section 26.15, for an orthogonal coordinate system with scale 
factors h; we have 


hn ifi=j, A lfhe ati Sy, 
8ij = : and gl= ° 
0 otherwise 0 otherwise, 


and so the determinant g of the matrix [g;;] is given by g = Afh3hi. 


Gradient 
The gradient of a scalar ¢ is given by 
: Od . 
Vo =. = oe (26.91) 


since the covariant derivative of a scalar is the same as its partial derivative. 


Divergence 
Replacing the partial derivatives that occur in Cartesian coordinates with covari- 
ant derivatives, the divergence of a vector field v in a general coordinate system 
is given by 
; ov! oon 
Viveuj= apt Pai : 
Using the expression (26.82) for the Christoffel symbol in terms of the metric 
tensor, we find 


Ogi OKI es) 1 O8iI (26.92) 


| Se Le 
Mania! (Set Sam gt) oe Sar 
The last two terms have cancelled because 


ORK OSI is OBR 


Ou aul & ul” 
where in the first equality we have interchanged the dummy indices i and /, and 
in the second equality have used the symmetry of the metric tensor. 
We may simplify (26.92) still further by using a result concerning the derivative 
of the determinant of a matrix whose elements are functions of the coordinates. 
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> Suppose A = [aj], B = [b] and that B= A~!. By considering the determinant a = |A\, 
show that 


0a ji ai; 


—— = «in? — 
ouk ouk 


If we denote the cofactor of the element a;; by A’ then the elements of the inverse matrix 
are given by (see chapter 8) 
3 tee 
bY = -A". (26.93) 
a 


However, the determinant of A is given by 
a= x aA", 
Jj 


in which we have fixed i and written the sum over j explicitly, for clarity. Partially 
differentiating both sides with respect to aj, we then obtain 


Lays (26.94) 


since aj; does not occur in any of the cofactors A’. 

Now, if the aj; depend on the coordinates then so will the determinant a and, by the 
chain rule, we have 

Ca da 0aij; ij Oi ap it Mii 

duk — Gay Ouk éuk éuk? 


in which we have used (26.93) and (26.94). < 


(26.95) 


Applying the result (26.95) to the determinant g of the metric tensor, and 
remembering both that gg; = 5/ and that g!/ is symmetric, we obtain 
og ij OB 

Substituting (26.96) into (26.92) we find that the expression for the Christoffel 
symbol can be much simplified to give 


fea 1 dg 1 O68 
Ki 2¢ duk Je ouk © 
Thus finally we obtain the expression for the divergence of a vector field in a 
general coordinate system as 


1 0 
JS dus 
Laplacian 


If we replace v by V# in V- vy then we obtain the Laplacian V7. From (26.91), 
we have 


Vive0j= 


(/gv!). (26.97) 
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and so the covariant components of v are given by v; = 0¢/éu!. In (26.97), 
however, we require the contravariant components v’. These may be obtained by 
raising the index using the metric tensor, to give 

; oo 

J = gtky = oJk 
vi =Zg uN =8 Buk’ 
Substituting this into (26.97) we obtain 
1 o 


. 
Papin Te jk OP 
Vo eau) (vee =). (26.98) 


> Use (26.98) to find the expression for V7¢ in an orthogonal coordinate system with scale 
factors h;, i = 1,2, 3. 


For an orthogonal coordinate system ./g = hyhyhs; further, gi =1/h ifi=jand gi =0 
otherwise. Therefore, from (26.98) we have 


1 0 [ hhh; 0d 
yas a 1LQN3 
¥ # hyhoh, ous ( he #) 


which agrees with the results of section 10.10. < 


Curl 
The special vector form of the curl of a vector field exists only in three dimensions. 
We therefore consider a more general form valid in higher-dimensional spaces as 
well. In a general space the operation curl v is defined by 
(curl V)ij = Vi; 7 — Vj; i, 


which is an antisymmetric covariant tensor. 
In fact the difference of derivatives can be simplified, since 


: AW. 
sp pea Sat OU aa 
ij bi Ba pel mA itl 
Ov; Ov; 
ous au’ 


where the Christoffel symbols have cancelled because of their symmetry properties. 
Thus curl v can be written in terms of partial derivatives as 
Ov; Ov; 
(curl v)jj = nat Bait 
Generalising slightly the discussion of section 26.17, in three dimensions we may 
associate with this antisymmetric second-order tensor a vector with contravariant 
components, 


‘ fs 33 
(Vxvji= 5 e!*(curl v) jx 
Spe ON cao ee, 
2/8 ouk Gus ve ous’ 
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this is the analogue of the expression in Cartesian coordinates discussed in 
section 26.8. 


26.21 Absolute derivatives along curves 


In section 26.19 we discussed how to differentiate a general tensor with respect 
to the coordinates and introduced the covariant derivative. In this section we 
consider the slightly different problem of calculating the derivative of a tensor 
along a curve r(t) that is parameterised by some variable t. 

Let us begin by considering the derivative of a vector v along the curve. If we 
introduce an arbitrary coordinate system u' with basis vectors e;, i = 1,2,3, then 
we may write v = v'e; and so obtain 


dv dvi ,de; 

aed dt 
_ di ; Oe; duk 
ae” Ok at’ 


here the chain rule has been used to rewrite the last term on the right-hand side. 
Using (26.75) to write the derivatives of the basis vectors in terms of Christoffel 
symbols, we obtain 

~ .duk 

J I 

yv e;. 

ik dt J 


dt = dt 
Interchanging the dummy indices i and j in the last term, we may factor out the 
basis vector and find 
dv dvi ; _;duk 
— =| — +I",v/— Je. 
dt (4 a er ) . 


The term in parentheses is called the absolute (or intrinsic) derivative of the 
components v’ along the curve r(t)and is usually denoted by 


bu! = dvi 


dv bu! ; du 
7 pei Ok Gp (26.99) 


Using the same method, the absolute derivative of the covariant components 
v; of a vector is given by 
Ov; du‘ 


TZ =H vVisk sz: 
ot 


dt 


Similarly, the absolute derivatives of the contravariant, mixed and covariant 
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components of a second-order tensor T are 


ore = Ti du 
otk dt’ 
oT’; _,  duk 
“or OK ae? 
6Ty _ 7 at 
Tae On 


The derivative of T along the curve r(t) may then be written in terms of, for 
example, its contravariant components as 
dT oT! duk 


— = —e @e; = T! , —e; @e;. 
dt aa fae! 


26.22 Geodesics 


As an example of the use of the absolute derivative, we conclude this chapter 
with a brief discussion of geodesics. A geodesic in real three-dimensional space 
is a straight line, which has two equivalent defining properties. Firstly, it is the 
curve of shortest length between two points and, secondly, it is the curve whose 
tangent vector always points in the same direction (along the line). Although 
in this chapter we have considered explicitly only our familiar three-dimensional 
space, much of the mathematical formalism developed can be generalised to more 
abstract spaces of higher dimensionality in which the familiar ideas of Euclidean 
geometry are no longer valid. It is often of interest to find geodesic curves in 
such spaces by using the defining properties of straight lines in Euclidean space. 

We shall not consider these more complicated spaces explicitly but will de- 
termine the equation that a geodesic in Euclidean three-dimensional space (1.e. 
a straight line) must satisfy, deriving it in a sufficiently general way that our 
method may be applied with little modification to finding the equations satisfied 
by geodesics in more abstract spaces. 

Let us consider a curve r(s), parameterised by the arc length s from some point 
on the curve, and choose as our defining property for a geodesic that its tangent 
vector t = dr/ds always points in the same direction everywhere on the curve, ie. 

dt 

a 0 (26.100) 
Alternatively, we could exploit the property that the distance between two points 
is a minimum along a geodesic and use the calculus of variations (see chapter 22); 
this would lead to the same final result (26.101). 

If we now introduce an arbitrary coordinate system u' with basis vectors e;, 
i= 1,2,3, then we may write t = t’e;, and from (26.99) we find 

dt duk 


Rogge” 
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Writing out the covariant derivative, we obtain 


dt ; _,duk 
(5 Ue a ) eae 


But, since t/ = du//ds, it follows that the equation satisfied by a geodesic is 
Pui ; dul duk 


> Find the equations satisfied by a geodesic (straight line) in cylindrical polar coordinates. 


From (26.83), the only non-zero Christoffel symbols are I'',, = —p and I?,, =T,, = 1/p. 
Thus the required geodesic equations are 


Cul, dw di &p dp \?* 
ag t Vn ds ds Y ds? ( ds ) e 
Pu du! du? Pp 2dpdd 
21” =0 0 
ds? con ds ds ds? + pds ds i 
us a@z 
= . <4 
ds? a ds? y 
26.23 Exercises 
26.1 Use the basic definition of a Cartesian tensor to show the following. 


(a) That for any general, but fixed, ¢, 
(u1, U2) = (x1 cos — x2 sind, x; sind + x2 cos f) 


are the components of a first-order tensor in two dimensions. 


(b) That 
x3 X1X2 
XX. XT 


is not a tensor of order 2. To establish that a single element does not 
transform correctly is sufficient. 


26.2 The components of two vectors, A and B, and a second-order tensor, T, are given 
in one coordinate system by 


1 0 2 3 0 
A=| 0], B={| 1]. T={ 8B 4 O 
0 0 0 0 2 
In a second coordinate system, obtained from the first by rotation, the components 


of A and B are 
if 3 i 
2 


; 0 
1 J3 
Find the components of T in this new coordinate system and hence evaluate, 
with a minimum of calculation, 


Ti Tji, Ti Tix Ty, Tix Tinn Thi Thm- 
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In section 26.3 the transformation matrix for a rotation of the coordinate axes 
was derived, and this approach is used in the rest of the chapter. An alternative 
view is that of taking the coordinate axes as fixed and rotating the components 
of the system; this is equivalent to reversing the signs of all rotation angles. 
Using this alternative view, determine the matrices representing (a) a positive 
rotation of 2/4 about the x-axis and (b) a rotation of —z/4 about the y-axis. 
Determine the initial vector r which, when subjected to (a) followed by (b), 
finishes at (3, 2, 1). 
Show how to decompose the Cartesian tensor T;; into three tensors, 


Ty = Uy + Vij + Sijp 


where Uj; is symmetric and has zero trace, Vj; is isotropic and S;; has only three 
independent components. 
Use the quotient law discussed in section 26.7 to show that the array 


yr? +22 — x? —2xy —2xz 
—2yx x? +27 — y? —2yz 
—2zx —2zy x? + y? — 2? 


forms a second-order tensor. 
Use tensor methods to establish the following vector identities: 


(a) (ux Vv) X w= (u- w)v—(v- wu; 

(b) curl(¢u) = ¢dcurlu+ (grad ¢) x u; 

(c) div(u x v) =v-curlu—u- curly; 

(d) curl(u x v) =(v- grad)u — (u- grad)v + udiv v — vdivu; 

(e) grad 5(u -u) =u X curlu+ (u- grad)u. 

Use result (e) of the previous question and the general divergence theorem for 
tensors to show that, for a vector field A, 


| [A(A - dS) — 447dS] = Fi [A divA — A x curl A] dV, 
S Vv 


where S is the surface enclosing volume V. 
A column matrix a has components a,,a,,a, and A is the matrix with elements 
Aij = —€ijnAk. 


(a) What is the relationship between column matrices b and c if Ab = c? 
(b) Find the eigenvalues of A and show that a is one of its eigenvectors. Explain 
why this must be so. 


Equation (26.29), 
|Alétnn = Aji AmjAnk€ijk, 


is a more general form of the expression (8.47) for the determinant of a 3 x 3 
matrix A. The latter could have been written as 


IA] = €1j An AjrAgs, 


whilst the former removes the explicit mention of 1,2,3 at the expense of an 
additional Levi—Civita symbol. As stated in the footnote on p. 942, (26.29) can 
be readily extended to cover a general N x N matrix. 

Use the form given in (26.29) to prove properties (i), (ili), (v), (vi) and (vii) 
of determinants stated in subsection 8.9.1. Property (iv) is obvious by inspection. 
For definiteness take N = 3, but convince yourself that your methods of proof 
would be valid for any positive integer N. 
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26.10 


26.11 


26.12 


26.13 


26.14 


26.15 


26.16 


A symmetric second-order Cartesian tensor is defined by 
Ti = Oj; — 3X;X;. 


Evaluate the following surface integrals, each taken over the surface of the unit 
sphere: 


(a) f tyass tb) f tetas (© f xTpas. 
Given a non-zero vector v, find the value that should be assigned to « to make 
Pij = QUj0; and Oi; = Oi =_— QUjVj 


into parallel and orthogonal projection tensors, respectively, i. tensors that 
satisfy, respectively, P;jvj = v;, Pu; = 0 and Q,jvj = 0, Qijuj = uj, for any vector 
u that is orthogonal to vy. 

Show, in particular, that Qj; is unique, ie. that if another tensor Tj; has the 
same properties as Q;; then (Qj; — Tjj)w; = 0 for any vector w. 
In four dimensions, define second-order antisymmetric tensors, Fj; and Q;;, and 
a first-order tensor, S;, as follows: 


(a) Fo = M1, Q23 = B, and their cyclic permutations; 
(b) Fi D;, Qis = E; for i= 1,2,3; 
(c) Sy =p, S; = J; for i= 1,2, 3. 


Then, taking x4, as t and the other symbols to have their usual meanings in 
electromagnetic theory, show that the equations yj OF; /0x; = S; and 0Q x. /0x;+ 
0Q4;/0xj + 6Q;;/0X; = 0 reproduce Maxwell’s equations. In the latter i, j,k is any 
set of three subscripts selected from 1, 2, 3, 4, but chosen in such a way that they 
are all different. 

In a certain crystal the unit cell can be taken as six identical atoms lying at the 
corners of a regular octahedron. Convince yourself that these atoms can also be 
considered as lying at the centres of the faces of a cube and hence that the crystal 
has cubic symmetry. Use this result to prove that the conductivity tensor for the 
crystal, oj;, must be isotropic. 

Assuming that the current density j and the electric field E appearing in equation 
(26.44) are first-order Cartesian tensors, show explicitly that the electrical con- 
ductivity tensor o;, transforms according to the law appropriate to a second-order 
tensor. 

The rate W at which energy is dissipated per unit volume, as a result of the 
current flow, is given by E-j. Determine the limits between which W must lie for 
a given value of |E| as the direction of E is varied. 

In a certain system of units, the electromagnetic stress tensor Mj; is given by 


Mi; = E;Ej + BiB; — $6ij(ExEx + BB), 


where the electric and magnetic fields, E and B, are first-order tensors. Show that 
M,; is a second-order tensor. 

Consider a situation in which |E| = |B|, but the directions of E and B are 
not parallel. Show that E+ B are principal axes of the stress tensor and find 
the corresponding principal values. Determine the third principal axis and its 
corresponding principal value. 

A rigid body consists of four particles of masses m, 2m, 3m, 4m, respectively 
situated at the points (a,a,a), (a,—a,—a), (—a,a,—a), (—a,—a,a) and connected 
together by a light framework. 


(a) Find the inertia tensor at the origin and show that the principal moments of 
inertia are 20ma? and (20 + 2,/5)ma?. 


979 


TENSORS 


26.17 


26.18 


26.19 


26.20 


26.21 


26.22 


(b) Find the principal axes and verify that they are orthogonal. 


A rigid body consists of eight particles, each of mass m, held together by light 
rods. In a certain coordinate frame the particles are at positions 


+a(3, 1,—1), a(l,—1,3),  ta(1,3,-1),  ta(—1, 1,3). 


Show that, when the body rotates about an axis through the origin, if the angular 


velocity and angular momentum vectors are parallel then their ratio must be 
40ma*, 64ma* or 72ma?. 


The paramagnetic tensor y;; of a body placed in a magnetic field, in which its 
energy density is —4 10M -H with M; = Sr %ijH;, is 


2k O 0 
O 3k k 
O k 3k 


Assuming depolarizing effects are negligible, find how the body will orientate 
itself if the field is horizontal, in the following circumstances: 


(a) the body can rotate freely; 
(b) the body is suspended with the (1,0,0) axis vertical; 
(c) the body is suspended with the (0, 1,0) axis vertical. 


A block of wood contains a number of thin soft-iron nails (of constant perme- 
ability). A unit magnetic field directed eastwards induces a magnetic moment in 
the block having components (3,1,—2), and similar fields directed northwards 
and vertically upwards induce moments (1,3,—2) and (—2,—2,2) respectively. 
Show that all the nails lie in parallel planes. 


For tin, the conductivity tensor is diagonal, with entries a, a, and b when referred 
to its crystal axes. A single crystal is grown in the shape of a long wire of length L 
and radius r, the axis of the wire making polar angle @ with respect to the crystal’s 
3-axis. Show that the resistance of the wire is L(nr?ab)—! (a cos” 6 + bsin’ 0). 


By considering an isotropic body subjected to a uniform hydrostatic pressure 
(no shearing stress), show that the bulk modulus k, defined by the ratio of the 
pressure to the fractional decrease in volume, is given by k = E/[3(1—20)] where 
E is Young’s modulus and a is Poisson’s ratio. 


For an isotropic elastic medium under dynamic stress, at time t the displacement 
u; and the stress tensor pj; satisfy 


Ou, Ou Opi Ou; 

if = Cin ( — and =p, 

aoe (= 7 i) ax; ? oP 

where cj; is the isotropic tensor given in equation (26.47) and p is a constant. 
Show that both V-u and V xu satisfy wave equations and find the corresponding 
wave speeds. 
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26.23 


26.24 


26.25 


26.26 


26.27 


A fourth-order tensor T;;x; has the properties 
Tjikt = —Tijk, Tijtk = —Tijki- 
Prove that for any such tensor there exists a second-order tensor K,,, such that 
Tijki = €jjm€kinKmn 


and give an explicit expression for K,,,. Consider two (separate) special cases, as 
follows. 


(a) Given that Tj) is isotropic and Tj; = 1, show that Tj: is uniquely deter- 
mined and express it in terms of Kronecker deltas. 
(b) If now Tj; has the additional property 


Thij = —Tijxt, 


show that Tj; has only three linearly independent components and find an 
expression for Tjj; in terms of the vector 


el 
Vi = — 7 jai Tijn- 


Working in cylindrical polar coordinates p,¢,z, parameterise the straight line 
(geodesic) joining (1,0,0) to (1,2/2,1) in terms of s, the distance along the line. 
Show by substitution that the geodesic equations, derived at the end of section 
26.22, are satisfied. 

In a general coordinate system uw’, i = 1,2,3, in three-dimensional Euclidean 
space, a volume element is given by 


dV = |e; du! - (e: du’ x e3 du’). 


Show that an alternative form for this expression, written in terms of the deter- 
minant g of the metric tensor, is given by 


dV = Jgdu' dw? du’. 


Show that, under a general coordinate transformation to a new coordinate 
system u"', the volume element dV remains unchanged, i.e. show that it is a scalar 
quantity. 

By writing down the expression for the square of the infinitesimal arc length (ds)? 
in spherical polar coordinates, find the components gj; of the metric tensor in this 
coordinate system. Hence, using (26.97), find the expression for the divergence 
of a vector field v in spherical polars. Calculate the Christoffel symbols (of the 
second kind) I", in this coordinate system. 

Find an expression for the second covariant derivative vj: jk = (vi.j);x of a vector 
v; (see (26.88)). By interchanging the order of differentiation and then subtracting 


the two expressions, we define the components Rig of the Riemann tensor as 


Ui: jk — Viskj = Ri gdh. 
Show that in a general coordinate system u! these components are given by 


; ar’, 
ik = Wy + r,t" 


I 
R 
ous ouk 


— m 1 
ijk ~ mj r gl mk: 


By first considering Cartesian coordinates, show that all the components R',,, = 0 
for any coordinate system in three-dimensional Euclidean space. 

In such a space, therefore, we may change the order of the covariant derivatives 
without changing the resulting expression. 


ijk 
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A curve r(t) is parameterised by a scalar variable t. Show that the length of the 
curve between two points, A and B, is given by 


L=[ jee du! dui 
Sia a 4 
Using the calculus of variations (see chapter 22), show that the curve r(t) that 
minimises L satisfies the equation 
Pui ri dui duk — 8dui 
dt? H dt dt 8 dt’ 
where s is the arc length along the curve, § = ds/dt and 8 = d’s/dt?. Hence, show 
that if the parameter ¢t is of the form t = as +b, where a and b are constants, 
then we recover the equation for a geodesic (26.101). 
[A parameter which, like ¢, is the sum of a linear transformation of s and a 
translation is called an affine parameter. ] 
We may define Christoffel symbols of the first kind by 


Die = gal: 
Show that these are given by 
1 (0gn | Ogi  Ogij 
Vij 3(33+ Bik Et). 


oul oul ouk 


By permuting indices, verify that 
e8in 
duk 


Using the fact that I’, =I, ,, show that 


= Dije + Uji. 


Sisk = 0, 


ie. that the covariant derivative of the metric tensor is identically zero in all 
coordinate systems. 


26.24 Hints and answers 


(a) uw} my cos} 0)— x2 sin(? 9), etc.; 

(b) a = 3? a) — 2sexyx +c? Ne £C x3 + 8x1 X. + scxjxX2 +s? Xi 

(a) (1/./2)(./2, 0, 0;0, 1,150, 1, 1). (b) (1/./2)(1, 0, -1;0, /2, 0; 1,0, 1). 

pe (209. Aes 2), 

Twice contract the array with the outer product of (x,y,z) with itself, thus 
obtaining the expression —(x? + y* + z?)?, which is an invariant and therefore a 
scalar. 

Write A;(0A;/0x;) as 0(A;A;)/0x; — Aj(0A;/0X)). 

(i) Write out the expression for |A™|, contract both sides of the equation with €jn, 
and pick out the expression for |A| on the RHS. Note that €jm,€j is a numerical 
scalar. 

(iii) Each non-zero term on the RHS contains any particular row index once and 
only once. The same can be said for the Levi—Civita symbol on the LHS. Thus 
interchanging two rows is equivalent to interchanging two of the subscripts of 
€imn, and thereby reversing its sign. Consequently, the magnitude of |A| remains 
the same but its sign is changed. 

(v) If, say, Ap; = 4Ap;, for some particular pair of values i and j and all p then, 
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26.11 


26.13 


26.15 


26.17 
26.19 
26.21 


26.23 


26.25 


26.27 


26.29 


in the (multiple) summation on the RHS, each A, appears multiplied by (with 
no summation over i and j) 


€ ijk At Amj + €jix Aj Ami = €ijk2-A1jAmj + €jikAlj2Amj = 9, 


since €jj = —€ jx. Consequently, grouped in this way all terms are zero and 
|A| = 0. 
(vi) Replace Ajj by Anj + AA); and note that AA;A)jA nei = 0 by virtue of 
result (v). 
(vii) If C = AB, 

|Cleimn = Aj By Amy By Anz B:k€ijk- 


Contract this with €;,, and show that the RHS is equal to €xyz|A™ |exyz |B]. It then 
follows from result (i) that |C| = |A||B]. 

a = |v|~*. Note that the most general vector has components w; = Jotul” +vu, 
where both u) and u® are orthogonal to y. 

Construct the orthogonal transformation matrix S for the symmetry operation 
of (say) a rotation of 2x/3 about a body diagonal and, setting L = S$"! = S', 
construct o’ = LoL! and require o’ = oc. Repeat the procedure for (say) a rotation 
of z/2 about the x3-axis. These together show that 01; = 022 = 033 and that 
all other o;; = 0. Further symmetry requirements do not provide any additional 
constraints. 

The transformation of 6;; has to be included; the principal values are +E - B. 
The third axis is in the direction +B x E with principal value —|E|’. 

The principal moments give the required ratios. 

The principal permeability, in direction (1, 1,2), has value 0. Thus all the nails lie 
in planes to which this is the normal. 

Take pit = p22 = p33 = —p, and pj = ej; = 0 for i # j, leading to —p = 
(A+2u/3)e;. The fractional volume change is e;; A and y are as defined in (26.46) 
and the worked example that follows it. 

Consider Qyq = €pij€qki Tijxi and Show that Ky, = Qn,/4 has the required property. 
(a) Argue from the isotropy of Tj; and €; for that of K,,, and hence that it 
must be a multiple of 6,,. Show that the multiplier is uniquely determined and 
that Tijx = (0:0 jk — 6;.6;1)/6. 

(b) By relabelling dummy subscripts and using the stated antisymmetry property, 
show that Ky = —Kyn. Show that —2V; = minK, and hence that Kin = €innVi- 
Tiki = xi Vi — €xj Vie 

Use |e; - (e2 X e3)| = JB. 

Recall that \/g” = |Gu/du'| /g and du!’ du’? dul? = \du' /du| du! du2 du. 

(0); ):k = (Wi: jp),k — We - VD visi and vj; ; = vi; — DP" jUm- If all components of a 
tensor equal zero in one coordinate system, then they are zero in all coordinate 
systems. 

Use gig!” = 6! and gi; = gj. Show that 


agii wae 
Bik = (4 — Dik — ra) ee 


and then use the earlier result. 
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Numerical methods 


It happens frequently that the end product of a calculation or piece of analysis 
is one or more algebraic or differential equations, or an integral that cannot be 
evaluated in closed form or in terms of tabulated or pre-programmed functions. 
From the point of view of the physical scientist or engineer, who needs numerical 
values for prediction or comparison with experiment, the calculation or analysis 
is thus incomplete. 

With the ready availability of standard packages on powerful computers for 
the numerical solution of equations, both algebraic and differential, and for the 
evaluation of integrals, in principle there is no need for the investigator to do 
anything other than turn to them. However, it should be a part of every engineer’s 
or scientist’s repertoire to have some understanding of the kinds of procedure that 
are being put into practice within those packages. The present chapter indicates 
(at a simple level) some of the ways in which analytically intractable problems 
can be tackled using numerical methods. 

In the restricted space available in a book of this nature, it is clearly not 
possible to give anything like a full discussion, even of the elementary points that 
will be made in this chapter. The limited objective adopted is that of explaining 
and illustrating by simple examples some of the basic principles involved. In 
many cases, the examples used can be solved in closed form anyway, but this 
‘obviousness’ of the answers should not detract from their illustrative usefulness, 
and it is hoped that their transparency will help the reader to appreciate some of 
the inner workings of the methods described. 

The student who proposes to study complicated sets of equations or make 
repeated use of the same procedures by, for example, writing computer programs 
to carry out the computations, will find it essential to acquire a good under- 
standing of topics hardly mentioned here. Amongst these are the sensitivity of 
the adopted procedures to errors introduced by the limited accuracy with which 
a numerical value can be stored in a computer (rounding errors) and to the 
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errors introduced as a result of approximations made in setting up the numerical 
procedures (truncation errors). For this scale of application, books specifically 
devoted to numerical analysis, data analysis and computer programming should 
be consulted. 

So far as is possible, the method of presentation here is that of indicating 
and discussing in a qualitative way the main steps in the procedure, and then 
of following this with an elementary worked example. The examples have been 
restricted in complexity to a level at which they can be carried out with a pocket 
calculator. Naturally it will not be possible for the student to check all the 
numerical values presented, unless he or she has a programmable calculator or 
computer readily available, and even then it might be tedious to do so. However, 
it is advisable to check the initial step and at least one step in the middle of 
each repetitive calculation given in the text, so that how the symbolic equations 
are used with actual numbers is understood. Clearly the intermediate step should 
be chosen to be at a point in the calculation at which the changes are still 
sufficiently large that they can be detected by whatever calculating device is 
used. 

Where alternative methods for solving the same type of problem are discussed, 
for example in finding the roots of a polynomial equation, we have usually 
taken the same example to illustrate each method. This could give the mistaken 
impression that the methods are very restricted in applicability, but it is felt by 
the authors that using the same examples repeatedly has sufficient advantages, in 
terms of illustrating the relative characteristics of competing methods, to justify 
doing so. Once the principles are clear, little is to be gained by using new 
examples each time, and, in fact, having some prior knowledge of the ‘correct 
answer’ should allow the reader to judge the efficiency and dangers of particular 
methods as the successive steps are followed through. 

One other point remains to be mentioned. Here, in contrast with every other 
chapter of this book, the value of a large selection of exercises is not clear cut. 
The reader with sufficient computing resources to tackle them can easily devise 
algebraic or differential equations to be solved, or functions to be integrated 
(which perhaps have arisen in other contexts). Further, the solutions of these 
problems will be self-checking, for the most part. Consequently, although a 
number of exercises are included, no attempt has been made to test the full range 
of ideas treated in this chapter. 


27.1 Algebraic and transcendental equations 


The problem of finding the real roots of an equation of the form f(x) = 0, where 
f(x) is an algebraic or transcendental function of x, is one that can sometimes 
be treated numerically, even if explicit solutions in closed form are not feasible. 
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10: f(x) = x5 —2x? 3 


x 


03 04 06 08 10 12 17 16 18 
—4t ii 


Figure 27.1 A graph of the function f(x) = x° — 2x? — 3 for x in the range 
O0<x<19. 


Examples of the types of equation mentioned are the quartic equation, 
ax* + bx +c =0, 

and the transcendental equation, 
x — 3tanhx =0. 


The latter type is characterised by the fact that it contains, in effect, a polynomial 
of infinite order on the LHS. 

We will discuss four methods that, in various circumstances, can be used to 
obtain the real roots of equations of the above types. In all cases we will take as 
the specific equation to be solved the fifth-order polynomial equation 


f(x) =x — 2x7 -3 =0. (27.1) 


The reasons for using the same equation each time were discussed in the intro- 
duction to this chapter. 

For future reference, and so that the reader may follow some of the calculations 
leading to the evaluation of the real root of (27.1), a graph of f(x) in the range 
0 <x < 1.9 is shown in figure 27.1. 

Equation (27.1) is one for which no solution can be found in closed form, that 
is in the form x = a, where a does not explicitly contain x. The general scheme to 
be employed will be an iterative one in which successive approximations to a real 
root of (27.1) will be obtained, each approximation, it is to be hoped, being better 
than the preceding one; certainly, we require that the approximations converge 
and that they have as their limit the sought-for root. Let us denote the required 
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root by € and the values of successive approximations by x1, x2, ..., Xn,--. . Then, 
for any particular method to be successful, 


lim x, =&, where f(€) =0. (27.2) 


no 


However, success as defined here is not the only criterion. Since, in practice, 
only a finite number of iterations will be possible, it is important that the values 
of x, be close to that of € for all n > N, where N is a relatively low number; 
exactly how low it is naturally depends on the computing resources available and 
the accuracy required in the final answer. 

So that the reader may assess the progress of the calculations that follow, we 
record that to nine significant figures the real root of equation (27.1) has the 
value 


€ = 1.495 106 40. (27.3) 


We now consider in turn four methods for determining the value of this root. 


27.1.1 Rearrangement of the equation 
If equation (27.1), f(x) = 0, can be recast into the form 
x = $(x), (27.4) 
where (x) is a slowly varying function of x, then an iteration scheme 


Xn = (Xn) (27.5) 


will often produce a fair approximation to the root & after a few iterations, 
as follows. Clearly, € = $(€), since f(€) = 0; thus, when x, is close to €, 
the next approximation, x,+41, will differ little from x,, the actual size of the 
difference giving an order-of-magnitude indication of the inaccuracy in Xn41 
(when compared with €). 

In the present case, the equation can be written 


x = (2x? + 3)!, (27.6) 


Because of the presence of the one-fifth power, the RHS is rather insensitive 
to the value of x used to compute it, and so the form (27.6) fits the general 
requirements for the method to work satisfactorily. It remains only to choose a 
starting approximation. It is easy to see from figure 27.1 that the value x = 1.5 
would be a good starting point, but, so that the behaviour of the procedure at 
values some way from the actual root can be studied, we will make a poorer 
choice, x; = 1.7. 
With this starting value and the general recurrence relationship 


Xing = (2x5 +3), (27.7) 
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Xn f (Xn) 
17 5.42 
1.54418 1.01 


1.506 86 2.28 x 107! 
1.49792 5.37 x 107? 
1.495 78 1.28 x 107? 
1.49527 3.11 x 103 
1.495 14 7.34 x 10-4 
1.49512 1.76 x 10-4 
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Table 27.1 Successive approximations to the root of (27.1) using the method 
of rearrangement. 


n A, f(An) Bn f (Bn) Xn f (Xn) 

1 10 —4.0000 1.7 5.4186 1.2973 —2.6916 
2 1.2973 —2.6916 1.7 5.4186 1.4310 —1.0957 
3 14310 —1.0957 1.7 5.4186 1.4762 —0.3482 
4 14762 —0.3482 1.7 5.4186 1.4897 —0.1016 
5 14897 —0.1016 1.7 5.4186 1.4936 —0.0289 
6 1.4936  —0.0289 1.7 5.4186 1.4947 —0.0082 


Table 27.2 Successive approximations to the root of (27.1) using linear 
interpolation. 


successive values can be found. These are recorded in table 27.1. Although not 
strictly necessary, the value of f(x,) = x> — 2x? — 3 is also shown at each stage. 

It will be seen that x7 and all later x, agree with the precise answer (27.3) to 
within one part in 104. However, f (xn) and x, — € are both reduced by a factor 
of only about 4 for each iteration; thus a large number of iterations would be 
needed to produce a very accurate answer. The factor 4 is, of course, specific 
to this particular problem and would be different for a different equation. The 
successive values of x, are shown in graph (a) of figure 27.2. 


27.1.2 Linear interpolation 


In this approach two values, A; and By, of x are chosen with A; < B,; and 
such that f(A1) and f(B;) have opposite signs. The chord joining the two points 
(Ai, f(A1)) and (Bi, f(B1)) is then notionally constructed, as illustrated in graph 
(b) of figure 27.2, and the value x; at which the chord cuts the x-axis is determined 
by the interpolation formula 


‘s Anf (Bn) _ Bnf (An) 
f(Bn)— f(An) ? 
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Figure 27.2 Graphical illustrations of the iteration methods discussed in 
the text: (a) rearrangement; (b) linear interpolation; (c) binary chopping; 
(d) Newton—Raphson. 


with n = 1. Next, f(x:) is evaluated and the process repeated after replacing 
either A; or By by x1, according to whether f(x;) has the same sign as f(A1) or 
f(B1), respectively. In figure 27.2(b), A; is the one replaced. 

As can be seen in the particular example that we are considering, with this 
method there is a tendency, if the curvature of f(x) is of constant sign near 
the root, for one of the two ends of the successive chords to remain un- 
changed. 

Starting with the initial values A; = 1 and B, = 1.7, the results of the first 
five iterations using (27.8) are given in table 27.2 and indicated in graph (b) of 
figure 27.2. As with the rearrangement method, the improvement in accuracy, 
as measured by f(x,) and x, — é, is a fairly constant factor at each iteration 
(approximately 3 in this case), and for our particular example there is little to 
choose between the two. Both tend to their limiting value of € monotonically, 
from either higher or lower values, and this makes it difficult to estimate limits 
within which € can safely be presumed to lie. The next method to be described 
gives at any stage a range of values within which € is known to lie. 
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An f(An) Bn f(Bn) Xn f (Xn) 

1.0000 —4.0000 1.7000 5.4186 1.3500 —2.1610 
1.3500 —2.1610 1.7000 5.4186 = 1.5250 0.5968 
1.3500 —2.1610 1.5250 0.5968 1.4375 —0.9946 
1.4375 —0.9946 1.5250 0.5968 1.4813 —0.2573 
1.4813 —0.2573 1.5250 0.5968 — 1.5031 0.1544 
1.4813 —0.2573 1.5031 0.1544 1.4922 —0.0552 
1.4922 —0.0552 1.5031 0.1544 1.4977 0.0487 
1.4922 —0.0552 1.4977 0.0487 1.4949 —0.0085 
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Table 27.3 Successive approximations to the root of (27.1) using binary 
chopping. 


27.1.3 Binary chopping 


Again two values of x, A, and By, that straddle the root are chosen, such that 
A; < B, and f(A;) and f(B;) have opposite signs. The interval between them is 
then halved by forming 


Xn = 5(An + Bn), (27.9) 


with n = 1, and f(x) is evaluated. It should be noted that x; is determined 
solely by A; and By, and not by the values of f(A1) and f(B;) as in the linear 
interpolation method. Now x; is used to replace either A; or Bi, depending on 
which of f(A;) or f(B,) has the same sign as f(x), i.e. if f(A) and f(x,) have the 
same sign then x; replaces A,. The process isthen repeated to obtain x, x3, etc. 

This has been carried through in table 27.3 for our standard equation (27.1) 
and is illustrated in figure 27.2(c). The entries have been rounded to four places 
of decimals. It is suggested that the reader follows through the sequential replace- 
ments of the A, and B, in the table and correlates the first few of these with 
graph (c) of figure 27.2. 

Clearly, the accuracy with which € is known in this approach increases by only 
a factor of 2 at each step, but this accuracy is predictable at the outset of the 
calculation and (unless f(x) has very violent behaviour near x = €) a range of x 
in which € lies can be safely stated at any stage. At the stage reached in the last 
row of table 27.3 it may be stated that 1.4949 < € < 1.4977. Thus binary chopping 
gives a simple approximation method (it involves less multiplication than linear 
interpolation, for example) that is predictable and relatively safe, although its 
convergence is slow. 


27.1.4 Newton—Raphson method 


The Newton—Raphson (NR) procedure is somewhat similar to the interpolation 
method, but, as will be seen, has one distinct advantage over the latter. Instead 
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is Xp f (Xn) 

Le 17 5.42 

2 1.545 01 1.03 

3 1.498 87 7.20 x 107? 
4 1.495 13 4.49 x 10-4 
5 1.49510640 2.6x 10° 
6 149510640 — 


Table 27.4 Successive approximations to the root of (27.1) using the Newton— 
Raphson method. 


of (notionally) constructing the chord between two points on the curve of f(x) 
against x, the tangent to the curve is notionally constructed at each successive 
value of x,, and the next value, x41, is taken as the point at which the tangent 
cuts the axis f(x) = 0. This is illustrated in graph (d) of figure 27.2. 

If the nth value is x,, the tangent to the curve of f(x) at that point has slope 
f'(Xn) and passes through the point x = x,, y = f(x,). Its equation is thus 


y(X) = (x — Xn) f' (Xn) + f (Xn). (27.10) 


The value of x at which y = 0 is then taken as x,,+1; thus the condition y(x,41) = 0 
yields, from (27.10), the iteration scheme 


= f (Xn) 
f'n) 

This is the Newton—Raphson iteration formula. Clearly, if x, is close to € then X41 

is close to x,, as it should be. It is also apparent that if any of the x, comes close 


(27.11) 


Xn+1 = Xn 


to a stationary point of f, so that f’(x,) is close to zero, the scheme is not going 
to work well. 
For our standard example, (27.11) becomes 


x3 — 2x? — 3 _ 4x3 — 2x2 +3 


eT 27.12 
5x4 — 4x) 5x4 — 4x, ( ) 


Xnt+l = Xn — 


Again taking a starting value of x; = 1.7, we obtain in succession the entries 
in table 27.4. The different values are given to an increasing number of decimal 
places as the calculation proceeds; f(x,) is also recorded. 

It is apparent that this method is unlike the previous ones in that the increase 
in accuracy of the answer is not constant throughout the iterations but improves 
dramatically as the required root is approached. Away from the root the behaviour 
of the series is less satisfactory, and from its geometrical interpretation it can be 
seen that if, for example, there were a maximum or minimum near the root then 
the series could oscillate between values on either side of it (instead of ‘homing 
in’ on the root). The reason for the good convergence near the root is discussed 
in the next section. 
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Of the four methods mentioned, no single one is ideal, and, in practice, some 
mixture of them is usually to be preferred. The particular combination of methods 
selected will depend a great deal on how easily the progress of the calculation 
may be monitored, but some combination of the first three methods mentioned, 
followed by the NR scheme if great accuracy were required, would be suitable 
for most situations. 


27.2 Convergence of iteration schemes 


For iteration schemes in which x,,4; can be expressed as a differentiable function 
of x,, for example the rearrangement or NR methods of the previous section, a 
partial analysis of the conditions necessary for a successful scheme can be made 
as follows. 

Suppose the general iteration formula is expressed as 


Xnt1 = F(Xn) (27.13) 

((27.7) and (27.12) are examples). Then the sequence of values x1, X2,...,Xy,--- 1S 
required to converge to the value € that satisfies both 

f()=90 and ¢=F(é). (27.14) 


If the error in the solution at the nth stage is ey, ie. X, = € + €,, then 


S + €nt1 = Xng1 = F(Xn) = F(C + €n). (27.15) 


For the iteration process to converge, a decreasing error is required, i.e. |én41| < 
|€,|. To see what this implies about F, we expand the right-hand term of (27.15) 
by means of a Taylor series and use (27.14) to replace (27.15) by 


Eten = E+ en (G+ 36 (C+. (27.16) 
This shows that, for small e,, 
ent © F'(E)en 
and that a necessary (but not sufficient) condition for convergence is that 
|F'(é)| <1. (27.17) 


It should be noted that this is a condition on F’(€) and not on f’(é), which 
may have any finite value. Figure 27.3 illustrates in a graphical way how the 
convergence proceeds for the case 0 < F’(é) < 1. 

Equation (27.16) suggests that if F(x) can be chosen so that F’(é) = 0 then the 
ratio |€,+1/€,| could be made very small, of order e, in fact. To go even further, 
if it can be arranged that the first few derivatives of F vanish at x = € then the 
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Figure 27.3 Illustration of the convergence of the iteration scheme x,4; = 
F(xn) when 0 < F’(é) < 1, where € = F(é). The line y = x makes an angle 
1/4 with the axes. The broken line makes an angle tan~! F’(é) with the x-axis. 


convergence, once x, has become close to €, could be very rapid indeed. If the 
first N — 1 derivatives of F vanish at x = €, ie. 


Fg) = FQ) = = FON) =O (27.18) 
and consequently 
Ent = O(€n), (27.19) 


then the scheme is said to have Nth-order convergence. 

This is the explanation of the significant difference in convergence between the 
NR scheme and the others discussed (judged by reference to (27.19), so that the 
differentiability of the function F is not a prerequisite). The NR procedure has 
second-order convergence, as is shown by the following analysis. Since 


_ f£Q) 

Fx)" 

_ Foo, flof"es 
'— F697 TROOP 
_ feof") 

[Foor 


F(x) =x 


F'(x) = 


Now, provided f’(é) # 0, it follows that F’(é) = 0 because f(x) =0 atx =€. 
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n Xn+1 En 

1 85 4.5 

2 5.191 1.19 

3 4.137 1.4 x 107! 
4 4.002257 2.3 x 10-3 
5 4.000000637 64x 1077 
6 4 — 


Table 27.5 Successive approximations to .\/16 using the iteration scheme 
(27.20). 


> The following is an iteration scheme for finding the square root of X: 


Xaet = 5 (« + =) (27.20) 


Xn 


Show that it has second-order convergence and illustrate its efficiency by finding, say, \/16 
starting with a very poor guess, \/16 = 1. 


If this scheme does converge to € then € will satisfy 


and so, since €* = X, 


whilst 


Thus the procedure has second-order, but not third-order, convergence. 

We now show the procedure in action. Table 27.5 gives successive values of x, and of 
€,, the difference between x, and the true value, 4. As we can see, the scheme is crude 
initially, but once x, gets close to €, it homes in on the true value extremely rapidly. < 


27.3 Simultaneous linear equations 


As we saw in chapter 8, many situations in physical science can be described 
approximately or exactly by a set of N simultaneous linear equations in N 
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variables (unknowns), x;, i = 1,2,..., N. The equations take the general form 


Aix, + Ajox2 +++» + Ainxn = by, 


Agi xX, + ApX2 + +++ + Awxn = bo, (27.21) 


Ayix1 + An2X2 +--+ + Annxn = by, 


where the A;; are constants and form the elements of a square matrix A. The b; 
are given and form a column matrix b. If A is non-singular then (27.21) can be 
solved for the x; using the inverse of A, according to the formula 


x =A'b. 


This approach was discussed at length in chapter 8 and will not be considered 
further here. 


27.3.1 Gaussian elimination 


We follow instead a continuation of one of the earliest techniques acquired by a 
student of algebra, namely the solving of simultaneous equations (initially only 
two in number) by the successive elimination of all the variables but one. This 
(known as Gaussian elimination) is achieved by using, at each stage, one of the 
equations to obtain an explicit expression for one of the remaining x; in terms 
of the others and then substituting for that x; in all other remaining equations. 
Eventually a single linear equation in just one of the unknowns is obtained. This 
is then solved and the result is resubstituted in previously derived equations (in 
reverse order) to establish values for all the xj. 

This method is probably very familiar to the reader, and so a specific example 
to illustrate this alone seems unnecessary. Instead, we will show how a calculation 
along such lines might be arranged so that the errors due to the inherent lack of 
precision in any calculating equipment do not become excessive. This can happen 
if the value of N is large and particularly (and we will merely state this) if the 
elements Aj, A22,...,Ann on the leading diagonal of the matrix in (27.21) are 
small compared with the off-diagonal elements. 

The process to be described is known as Gaussian elimination with interchange. 
The only, but essential, difference from straightforward elimination is that before 
each variable x; is eliminated, the equations are reordered to put the largest (in 
modulus) remaining coefficient of x; on the leading diagonal. 

We will take as an illustration a straightforward three-variable example, which 
can in fact be solved perfectly well without any interchange since, with simple 
numbers and only two eliminations to perform, rounding errors do not have 
a chance to build up. However, the important thing is that the reader should 
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appreciate how this would apply in (say) a computer program for a 1000-variable 
case, perhaps with unforseeable zeros or very small numbers appearing on the 
leading diagonal. 


> Solve the simultaneous equations 


— 
S 
es) 
& 
| 
NX 
=) 
tad 
oc 
+ 
tad 
o 
| 
= 
N 


(27.22) 


Firstly, we interchange rows (a) and (b) to bring the term 3x, onto the leading diagonal. In 
the following, we label the important equations (1), (IJ), (II), and the others alphabetically. 
A general (i.e. variable) label will be denoted by j. 


(I) 3x1 —20x2 +X3 = 12, 
(d) x1 +6x —4x3 = 8, 
(e) —x1 +3x2. +5x3 = 3. 
For (j) = (d) and (e), replace row (j) by 
row (j)— e x row (I), 
where aj; is the coefficient of x; in row (j), to give the two equations 
(I) (6+¥)u +(-4-Hm = 8-% 
() GB +544) = 34% 


Now |6 + 2| > (3- 2 and so no interchange is required before the next elimination. To 
eliminate x, replace row (f) by 
u 
(=3) 


38 


3 


row (f) — x row (II). 


This gives 
(IIT) [P+ x xy =748 «4 
Collecting together and tidying up the final equations, we have 
(1) 3x1 —20x +X3 = 12, 
(ID) 38x. —13x3 = 12, 
(IIL) x, = 2. 
Starting with (III) and working backwards, it is now a simple matter to obtain 


x = 10, x= 1, x3=2.<4 


27.3.2 Gauss-Seidel iteration 


In the example considered in the previous subsection an explicit way of solving 
a set of simultaneous equations was given, the accuracy obtainable being limited 
only by the rounding errors in the calculating facilities available, and the calcula- 
tion was planned to minimise these. However, in some situations it may be that 
only an approximate solution is needed. If, for a large number of variables, this is 
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the case then an iterative method may produce a satisfactory degree of precision 
with less calculation. Such a method, known as Gauss-Seidel iteration, is based 
upon the following analysis. 

The problem is again that of finding the components of the column matrix x 
that satisfies 


Ax =b (27.23) 


when A and b are a given matrix and column matrix, respectively. 
The steps of the Gauss-Seidel scheme are as follows. 


(i) 


(il) 


(iii) 


Rearrange the equations (usually by simple division on both sides of each 
equation) so that all diagonal elements of the new matrix C are unity, i.e. 
(27.23) becomes 


Cx = d, (27.24) 
where C =!—F, and F has zeros as its diagonal elements. 
Step (i) produces 
Fx+d=Ix=x, (27.25) 


and this forms the basis of an iteration scheme, 
Xn41 = FX, +d, (27.26) 


where x, is the nth approximation to the required solution vector €. 

To improve the convergence, the matrix F, which has zeros on its leading 
diagonal, can be written as the sum of two matrices L and U that have 
non-zero elements only below and above the leading diagonal, respectively: 


Fij ifi> db 
Li = 
0 otherwise, 


(27.27) 


Fij ifi< JI 
Uip= 4 : 
0 otherwise. 


This allows the latest values of the components of x to be used at each 
stage and an improved form of (27.26) to be obtained: 


Xnt1 = LXp41 + Ux, +d. (27.28) 


To see why this is possible, we note, for example, that when calculating, 
say, the fourth component of x,41, its first three components are already 
known, and, because of the structure of L, these are the only ones needed 
to evaluate the fourth component of Lx,41. 
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X1 x2 x3 
2 2 2 
4 0.1 1.34 


12.76 1.381 2.323 
9.008 0.867 1.881 
10.321 1.042 2.039 
9.902 0.987 1.988 
10.029 1.004 2.004 
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Table 27.6 Successive approximations to the solution of simultaneous equa- 
tions (27.29) using the Gauss-Seidel iteration method. 


> Obtain an approximate solution to the simultaneous equations 


x1 +6x> —4x3 = 8, 
3x1 —20x2 +x3 12, 


SN +3x2 +5x3 3: 


These are the same equations as were solved in subsection 27.3.1. 


Divide the equations by 1, —20 and 5, respectively, to give 
xX, + 6x2 — 4x3 = 8, 
—0.15x; + x. — 0.05x3 = —0.6, 
—0.2x; + 0.6x2 + x3 = 0.6. 


Thus, set out in matrix form, (27.28) is, in this case, given by 


x1 0 0 0 X{ 

xX2 => 0.15 0 0 x2 

‘i 02 —06 0 x3 
n+l n+l 


0 -6 4 xy 8 
+1 0 0 - 0.05 Xp +), —0.6 }. 
0 0 0 x3 J, 0.6 


Suppose initially (n = 1) we guess each component to have the value 2. Then the successive 
sets of values of the three quantities generated by this scheme are as shown in table 27.6. 
Even with the rather poor initial guess, a close approximation to the exact result, x; = 10, 
X72 = 1, x3 = 2, is obtained in only a few iterations. < 


27.3.3 Tridiagonal matrices 


Although for the solution of most matrix equations Ax = b the number of 
operations required increases rapidly with the size N x N of the matrix (roughly 
as N°), for one particularly simple kind of matrix the computing required increases 
only linearly with N. This type often occurs in physical situations in which objects 
in an ordered set interact only with their nearest neighbours and is one in which 
only the leading diagonal and the diagonals immediately above and below it 
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contain non-zero entries. Such matrices are known as tridiagonal matrices. They 
may also be used in numerical approximations to the solutions of certain types 
of differential equation. 

A typical matrix equation involving a tridiagonal matrix is as follows: 


by cy 0 ma yy 


dy by Cy Xy V2 
as b, C3 x3 ¥3 (27.30) 
ay) by Ov Xy-1 YN-1 


0 dy by 


So as to keep the entries in the matrix as free from subscripts as possible, we 
have used a, b and c to indicate subdiagonal, leading diagonal and superdiagonal 
elements, respectively. As a consequence, we have had to change the notation for 
the column matrix on the RHS from b to (say) y. 

In such an equation the first and last rows involve x; and xy, respectively, and 
so the solution could be found by letting x; be unknown and then solving in turn 
each row of the equation in terms of x;, and finally determining x; by requiring 
the next-to-last line to generate for xy an equation compatible with that given 
by the last line. However, if the matrix is large this becomes a very cumbersome 
operation, and a simpler method is to assume a form of solution 


Xi-1 = O;-1x1 + Gi-1. (27.31) 
Since the ith line of the matrix equation is 
ajyXj—1 + bX; + CiXi41 = Vis 
we must have, by substituting for x;_1, that 
(aj0;-1 + bi)xi + CiXi41 = Vi — GiGi-1. 


This is also in the form of (27.31), but with i replaced by i+ 1. Thus the recurrence 
formulae for 0; and ¢; are 


Ti Vi = GiGi 
0; = : = : 27.32 
iO. +8; e aj0j-1 + bj 
provided the denominator does not vanish for any i. From the first of the matrix 
equations it follows that 0; = —c,/b; and ¢, = y;/b;. The equations may now 


be solved for the x; in two stages without carrying through an unknown quantity. 
First, all the 0; and ¢; are generated using (27.32) and the values of 0; and ¢4, 
then, as a second stage, (27.31) is used to evaluate the x;, starting with xy (= oy) 
and working backwards. 
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Solve the following tridiagonal matrix equation, in which only non-zero elements are 
shown: 


The solution is set out in table 27.7, in which the arrows indicate the general flow of the 
calculation. First, the columns of a;, b;, c; and y; are filled in from the original equation 
(27.33) and then the recurrence relations (27.32) are used to fill in the successive rows 
starting at the top; on each row we work from left to right as far as and including the ¢; 
column. Finally, the bottom entry in the the x; column is set equal to the bottom entry in 
the completed ¢; column and the rest of the x; column is completed by using (27.31) and 


working up from the bottom. Thus the solution is x; = 2; x2 = 1; x3 = 3; x4 1s Xs 

qj be Gi a O14 + 5; 6; 

iy 0 1 20> 1 —2 

1 -1 2 1 —- 4 —1/4 7/4 

1 2 --1 2 > —3/2 4/3 7/2 13/3 

J 3 1 1 —- 5 -1/5 13 —3/5 

1 3 4 20> 17/5 —10/17 —9/5 44/17 

1 —2 2 0 -— 54/17 0 —88/17 17> 


Table 27.7. The solution of tridiagonal matrix equation (27.33). The arrows 
indicate the general flow of the calculation, as described in the text. 


2;x5=1.< 


27.4 Numerical integration 


As noted at the start of this chapter, with modern computers and computer 
packages — some of which will present solutions in algebraic form, where that 
is possible — the inability to find a closed-form expression for an integral no 
longer presents a problem. But, just as for the solution of algebraic equations, it 
is extremely important that scientists and engineers should have some idea of the 
procedures on which such packages are based. In this section we discuss some of 
the more elementary methods used to evaluate integrals numerically and at the 
same time indicate the basis of more sophisticated procedures. 
The standard integral evaluation has the form 


b 
I= i, f(x) dx, (27.34) 


where the integrand f(x) may be given in analytic or tabulated form, but for the 
cases under consideration no closed-form expression for J can be obtained. All 
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(a) (b) (c) 
fist 
fi fist 
= h—=> h 
| 
xi Xi41/2 Xi+1 Xj Xi+1 Xi-1 Xi Xi+1 


Figure 27.4 (a) Definition of nomenclature. (b) The approximation in using 
the trapezium rule; f(x) is indicated by the broken curve. (c) Simpson’s rule 
approximation; f(x) is indicated by the broken curve. The solid curve is part 
of the approximating parabola. 


numerical evaluations of I are based on regarding I as the area under the curve 
of f(x) between the limits x = a and x = b and attempting to estimate that area. 

The simplest methods of doing this involve dividing up the intervala <x <b 
into N equal sections, each of length h = (b—a)/N. The dividing points are 
labelled x;, with xo = a, xy = b, i running from 0 to N. The point x; is a distance 
ih from a. The central value of x in a strip (x = x; + h/2) is denoted for brevity 
by Xi+1/2, and for the same reason f(x;) is written as f;. This nomenclature is 
indicated graphically in figure 27.4(a). 

So that we may compare later estimates of the area under the curve with the 
true value, we next obtain an exact expression for I, even though we cannot 
evaluate it. To do this we need to consider only one strip, say that between x; 
and x;41. For this strip the area is, using Taylor’s expansion, 


h/2 


Fern +y)dy -[" i Oxia) 


h/2 mo 


h/2 
(i y" 
= ria fi, al dy 


h n+1 
= 257 a (3) (27.35) 


n even 


It should be noted that, in this exact expression, only the even derivatives of 
f survive the integration and all derivatives are evaluated at x;41/2. Clearly 
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other exact expressions are possible, e.g. the integral of f(x; + y) over the range 
0<y<h, but we will find (27.35) the most useful for our purposes. 

Although the preceding discussion has implicitly assumed that both of the 
limits a and b are finite, with the consequence that N is finite, the general method 
can be adapted to treat some cases in which one of the limits is infinite. It is 
sufficient to consider one infinite limit, as an integral with limits —oo and oo can 
be considered as the sum of two integrals, each with one infinite limit. 


Consider the integral 
T= i “fixax, 


where a is chosen large enough that the integrand is monotonically decreasing 
for x > a and falls off more quickly than x~?. The change of variable t = 1/x 


converts this integral into 
ee ee fe! 
T= sf {—)} dt. 
fe) 


It is now an integral over a finite range and the methods indicated earlier can be 
applied to it. The value of the integrand at the lower end of the t-range is zero. 

In a similar vein, integrals with an upper limit of oo and an integrand that is 
known to behave asymptotically as g(x)e-**, where g(x) is a smooth function, can 
be converted into an integral over a finite range by setting x = —«! Inat. Again, 
the lower limit, a, for this part of the integral should be positive and chosen 
beyond the last turning point of g(x). The part of the integral for x < a is treated 
in the normal way. However, it should be added that if the asymptotic form 
of the integrand is known to be a linear or quadratic (decreasing) exponential 
then there are better ways of estimating it numerically; these are discussed in 
subsection 27.4.3 on Gaussian integration. 

We now turn to practical ways of approximating I, given the values of fi, or a 
means to calculate them, for i=0,1,...,.N. 


27.4.1 Trapezium rule 


In this simple case the area shown in figure 27.4(a) is approximated as shown in 
figure 27.4(b), ic. by a trapezium. The area A; of the trapezium is 


Aj = 5(fi + fish, (27.36) 
and if such contributions from all strips are added together then the estimate of 


the total, and hence of J, is 


N-1 
I(estim.) = S~ A; = (fo pt Reiys ~<a 
i=0 
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This provides a very simple expression for estimating integral (27.34); its accuracy 
is limited only by the extent to which h can be made very small (and hence N 
very large) without making the calculation excessively long. Clearly the estimate 
provided is exact only if f(x) is a linear function of x. 

The error made in calculating the area of the strip when the trapezium rule is 
used may be estimated as follows. The values used are f; and fj41, as in (27.36). 
These can be expressed accurately in terms of fj,1/. and its derivatives by the 
Taylor series 


h. 1 (h\?, (i a ae 
fieraiy2 = Fisay2 = 5fisay2 ty (5) fupt ay (3) foyatin- 
Thus 


Aj(estim.) = $h(f; + fi+1) 


. 1 h I 4 
=h fini + 5 fis. +O) 


’ 


whilst, from the first few terms of the exact result (27.35), 


3 
Aj(exact) = hfizij2 + = G) fis. + O(0). 
Thus the error AA; = A;(estim.) — A;(exact) is given by 
AA, = (5 — 34) Wty + O(h°) 
~ BI fia 


The total error in [(estim.) is thus given approximately by 
Al(estim.) = nh? (f") = £(b- ah (f"), (27.38) 


where (f”) represents an average value for the second derivative of f over the 
interval a to b. 


> Use the trapezium rule with h = 0.5 to evaluate 


2 
r= f (x? — 3x + 4) dx, 
0 


and, by evaluating the integral exactly, examine how well (27.38) estimates the error. 


With h = 0.5, we will need five values of f(x) = x? — 3x44 for use in formula (27.37). 
They are f(0) = 4, f(0.5) = 2.75, f(1) = 2, f(1.5) = 1.75 and f(2) = 2. Putting these into 
(27.37) gives 


I(estim.) = oO +2x275 + 2x24 2x 175 + 2)=4.75. 


The exact value is 
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The difference between the estimate of the integral and the exact answer is 1/12. Equation 
(27.38) estimates this error as 2 x 0.25 x (f”)/12. Our (deliberately chosen!) integrand is 
one for which (f”) can be evaluated trivially. Because f(x) is a quadratic function of x, 
its second derivative is constant, and equal to 2 in this case. Thus (f”) has value 2 and 
(27.38) estimates the error as 1/12; that the estimate is exactly right should be no surprise 
since the Taylor expansion for a quadratic polynomial about any point always terminates 
after three terms and so no higher-order terms in h have been ignored in (27.38). < 


27.4.2 Simpson’s rule 


Whereas the trapezium rule makes a linear interpolation of f, Simpson’s rule 
effectively mimics the local variation of f(x) using parabolas. The strips are 
treated two at a time (figure 27.4(c)) and therefore their number, N, should be 
made even. 

In the neighbourhood of x;, for i odd, it is supposed that f(x) can be adequately 
represented by a quadratic form, 


f(xity) =fitay + by’. (27.39) 
In particular, applying this to y = +h yields two expressions involving b 
fier =f (xi +h) =fitah+br’, 
fi =f(xi-—h) = fi—ah+ bh’; 


thus 
bh? = 43(fia1 + fit — 2f)). 
Now, in the representation (27.39), the area of the double strip from x;_1 to 
Xi41 1S given by 


h 
Ajestim.) = | (fi + ay + by’) dy = 2hf;+ 3bh. 
—h 


Substituting for bh? then yields, for the estimated area, 
Aj(estim.) = 2hfi + zh x s(fivt + fi-t = 2fi) 
= $h(4fit fia + fi-1)s 
an expression involving only given quantities. It should be noted that the values 


of neither b nor a need be calculated. 
For the full integral, 


I(estim.) = $h (x +fn+4 ye fin +2 a in ; (27.40) 


m odd m even 


It can be shown, by following the same procedure as in the trapezium rule case, 
that the error in the estimated area is approximately 


(b = a) nif). 


Al(estim.) ~ 180 
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27.4.3, Gaussian integration 


In the cases considered in the previous two subsections, the function f was 
mimicked by linear and quadratic functions. These yield exact answers if f 
itself is a linear or quadratic function (respectively) of x. This process could 
be continued by increasing the order of the polynomial mimicking-function so 
as to increase the accuracy with which more complicated functions f could be 
numerically integrated. However, the same effect can be achieved with less effort 
by not insisting upon equally spaced points x;. 

The detailed analysis of such methods of numerical integration, in which the 
integration points are not equally spaced and the weightings given to the values at 
each point do not fall into a few simple groups, is too long to be given in full here. 
Suffice it to say that the methods are based upon mimicking the given function 
with a weighted sum of mutually orthogonal polynomials. The polynomials, F,,(x), 
are chosen to be orthogonal with respect to a particular weight function w(x), Le. 


b 
/ F(x) a(x) W(X) dx = KeySms 


where k,, is some constant that may depend upon n. Often the weight function is 
unity and the polynomials are mutually orthogonal in the most straightforward 
sense; this is the case for Gauss—Legendre integration for which the appropriate 
polynomials are the Legendre polynomials, P,(x). This particular scheme is 
discussed in more detail below. 

Other schemes cover cases in which one or both of the integral limits a and b 
are not finite. For example, if the limits are 0 and oo and the integrand contains 
a negative exponential function e~**, a simple change of variable can cast it 
into a form for which Gauss—Laguerre integration would be particularly well 
suited. This form of quadrature is based upon the Laguerre polynomials, for 
which the appropriate weight function is w(x) = e-*. Advantage is taken of this, 
and the handling of the exponential factor in the integrand is effectively carried 
out analytically. If the other factors in the integrand can be well mimicked by 
low-order polynomials, then a Gauss—Laguerre integration using only a modest 
number of points gives accurate results. 

If we also add that the integral over the range —co to oo of an integrand 
containing an explicit factor exp(—fBx?) may be conveniently calculated using a 
scheme based on the Hermite polynomials, the reader will appreciate the close 
connection between the various Gaussian quadrature schemes and the sets of 
eigenfunctions discussed in chapter 18. As noted above, the Gauss—Legendre 
scheme, which we discuss next, is just such a scheme, though its weight function, 
being unity throughout the range, is not explicitly displayed in the integrand. 

Gauss—Legendre quadrature can be applied to integrals over any finite range 
though the Legendre polynomials P/(x) on which it is based are only defined 
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and orthogonal over the interval —1 < x < 1, as discussed in subsection 18.1.2. 
Therefore, in order to use their properties, the integral between limits a and b in 
(27.34) has to be changed to one between the limits —1 and +1. This is easily 
done with a change of variable from x to z given by 
2x—b—a 
z= a tae) 


b—a 


so that I becomes 


aa 1 
pote i g(z)dz, (27.41) 
4 


in which g(z) = f(x). 

The n integration points x; for an n-point Gauss—Legendre integration are 
given by the zeros of P,,(x), i.e. the x; are such that P,,(x;) = 0. The integrand g(x) 
is mimicked by the (n — 1)th-degree polynomial 


n 


P(x) 
G oo Fin =a ADIT LA: i)> 
0) = 0 epPiey 8 
which coincides with g(x) at each of the points x;, i = 1,2,...,n. To see this it 
should be noted that 
: P,(Xx) 
lim. ——————. = 0x. 
rom (= x)PAO) 
It then follows, to the extent that g(x) is well reproduced by G(x), that 


n 


1 
i. g(x) dx ~ S~ a(x) fo Pre) gy. (27.42) 
= 


i=1 Pi (xi) -1X¥— Xj 


The expression 


1 ' P(x) 
we Pi (xi) ie xX — Xj a 


can be shown, using the properties of Legendre polynomials, to be equal to 


2 
(1 = x7)/Pa(Qxi)?” 


Wi = 


which is thus the weighting to be attached to the factor g(x;) in the sum (27.42). 
The latter then becomes 


1 n 
i g(x) dx ~ S> wig(xi). (27.43) 
= i=1 


In fact, because of the particular properties of Legendre polynomials, it can be 
shown that (27.43) integrates exactly any polynomial of degree up to 2n— 1. The 
error in the approximate equality is of the order of the 2nth derivative of g, and 
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so, provided g(x) is a reasonably smooth function, the approximation is a good 
one. 

Taking 3-point integration as an example, the three x; are the zeros of P3(x) = 
(5x3 — 3x), namely 0 and +0.774 60, and the corresponding weights are 


2 8 2 2) 


a d ’ 
ices 2 =. (1—0.6)x ($)* 9 


Table 27.8 gives the integration points (in the range —1 < x; < 1) and the 
corresponding weights w; for a selection of n-point Gauss—Legendre schemes. 


> Using a 3-point formula in each case, evaluate the integral 


yal 
r= Teg 


(i) using the trapezium rule, (11) using Simpson’s rule, (iii) using Gaussian integration. Also 
evaluate the integral analytically and compare the results. 


(i) Using the trapezium rule, we obtain 


T=3%x5[fO+2f (3) +f] 
[1+ $+ 4] = 0.7750. 


L 
2 
1 
q 
(ii) Using Simpson’s rule, we obtain 
r=tx![f0) +4f (4) + £0) 

= % [1+ $43] = 0.7833. 


(iii) Using Gaussian integration, we obtain 
pe a! if dz 
2 Ji 1t+i(z2+1/P 


=} {0.555 56 [ f (—0.774 60) + f(0.774 60)] + 0.888 89f(0)} 


=! {0.555 56 [0.987 458 +. 0.559 503] + 0.888 89 x os} 
= 0.785 27. 
(iv) Exact evaluation gives 
' dx 1 7 
I= | —, = [tan x], = = = 0.78540. 
| 142 [tan™' x], mn 
In practice, a compromise has to be struck between the accuracy of the result achieved 


and the calculational labour that goes into obtaining it. < 


Further Gaussian quadrature procedures, ones that utilise the properties of the 
Chebyshev polynomials, are available for integrals over finite ranges when the 
integrands involve factors of the form (1 —x?)+!/?. In the same way as decreasing 
linear and quadratic exponentials are handled through the weight functions in 
Gauss—Laguerre and Gauss—Hermite quadrature, respectively, the square root 
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Gauss—Legendre integration 
1 n 
[ ferde=S mises) 
i i=1 


n=9 

0.00000 00000 0.33023 93550 
0.32425 34234  0.3123470770 
0.61337 14327 0.26061 06964 
0.83603 11073 0.18064 81607 
0.9681602395  0.0812743884 


n=2 — 
0.57735 02692 1.0000000000 


n=3 
0.0000000000 0.88888 88889 
0.77459 66692 0.55555 55556 


n=4 
0.33998 10436 0.6521451549 
0.8611363116 0.34785 48451 


n= 10 

0.14887 43390  0.2955242247 
0.43339 53941 0.26926 67193 
0.67940 95683 0.21908 63625 
0.86506 33667 0.14945 13492 
0.97390 65285 0.06667 13443 


n=s 

0.00000 00000 0.56888 88889 
0.5384693101 0.47862 86705 
0.9061798459 0.23692 68851 |} n= 12 

0.12523 34085 0.24914 70458 
0.36783 14990 0.23349 25365 
0.58731 79543 0.20316 74267 
0.76990 26742 0.16007 83285 
0.90411 72564 0.10693 93260 
0.98156 06342  0.0471753364 


n=6 

0.2386191861 0.46791 39346 
0.6612093865 0.36076 15730 
0.9324695142 0.1713244924 


n=7 

0.00000 00000 0.41795 91837 
0.4058451514 0.3818300505 
0.7415311856 0.27970 53915 
0.94910 79123 0.12948 49662 


n= 20 

0.07652 65211 0.15275 33871 
0.22778 58511  0.1491729865 
0.37370 60887 0.14209 61093 
0.51086 70020 0.13168 86384 
0.63605 36807 ~—-0.11819 45320 
0.74633 19065 0.10193 01198 
0.8391169718 0.08327 67416 
0.91223 44283 0.06267 20483 
0.96397 19272 0.04060 14298 
0.99312 85992 0.01761 40071 


n=8 

0.18343 46425 0.36268 37834 
0.52553 24099 0.31370 66459 
0.79666 64774 0.22238 10345 
0.96028 98565 0.10122 85363 


Table 27.8 The integration points and weights for a number of n-point 
Gauss—Legendre integration formulae. The points are given as +x; and the 
contributions from both +x; and —x; must be included. However, the contri- 
bution from any point x; = 0 must be counted only once. 
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factor is treated accurately in Gauss—Chebyshev integration. Thus 


BY ey. 

1J/1—x? 
where the integration points x; are the zeros of the Chebyshev polynomials of 
the first kind T,,(x) and w; are the corresponding weights. Fortunately, both sets 


dx = s wif (xi), (27.44) 


are analytic and can be written compactly for all n as 


a oxsal 
| 
estes = wi=* fer adie (27.45) 


Note that, for any given n, all points are weighted equally and that no special 
action is required to deal with the integrable singularities at x = +1; they are 
dealt with automatically through the weight function. 

For integrals involving factors of the form (1—x?)!/?, the corresponding formula, 
based on Chebyshev polynomials of the second kind U,,(x), is 


ie f(x)V1 — x? dx = - wif (Xi). (27.46) 
i=1 
with integration points and weights given, for i= 1,... ,n, by 
in T : in 
B= EOS ag agra ora (27.47) 
For discussions of the many other schemes available, as well as their relative 
merits, the reader is referred to books devoted specifically to the theory of 


numerical analysis. There, details of integration points and weights, as well as 
quantitative estimates of the error involved in replacing an integral by a finite 
sum, will be found. Table 27.9 gives the points and weights for a selection of 
Gauss—Laguerre and Gauss-Hermite schemes.S 


27.4.4 Monte Carlo methods 


Surprising as it may at first seem, random numbers may be used to carry out 
numerical integration. The random element comes in principally when selecting 
the points at which the integrand is evaluated, and naturally does not extend to 
the actual values of the integrand! 

For the most part we will continue to use as our model one-dimensional 
integrals between finite limits, as typified by equation (27.34). Extensions to cover 
infinite or multidimensional integrals will be indicated briefly at the end of the 
section. It should be noted here, however, that Monte Carlo methods — the name 


S They, and those presented in table 27.8 for Gauss—Legendre integration, are taken from the much 
more comprehensive sets to be found in M. Abramowitz and I. A. Stegun (eds), Handbook of 
Mathematical Functions (New York: Dover, 1965). 
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Gauss—Laguerre and Gauss—Hermite integration 


[ crea=Swtey fe forde= mses) 
i=1 a i=l 


Xj Wi 
n=2 

0.58578 64376 0.85355 33906 
3.41421 35624 0.14644 66094 


n=2 
0.7071067812 0.88622 69255 


n=3 
0.0000000000 1.18163 59006 
1.2247448714 0.29540 89752 


n=3 

0.4157745568 0.71109 30099 
2.29428 03603 0.27851 77336 
6.28994 50829 0.01038 92565 n=4 

0.52464 76233 0.80491 40900 
n=4 1.65068 01239 0.08131 28354 
0.32254 76896 0.60315 41043 
1.7457611012 0.35741 86924 
4.5366202969 0.03888 79085 


9.3950709123 0.00053 92947 


n=5 

0.0000000000 0.94530 87205 
0.95857 24646 0.39361 93232 
2.02018 28705 0.01995 32421 
n=5 

0.2635603197 0.52175 56106 
1.4134030591 0.39866 68111 
3.5964257710 0.07594 24497 
7.0858100059 0.00361 17587 
12.6408 00844 0.00002 33700 


n=6 

0.43607 74119 0.7246295952 
1.3358490740 0.15706 73203 
2.3506049737 0.00453 00099 


n=7 

0.0000000000 0.81026 46176 
0.81628 78829  0.4256072526 
1.67355 16288 0.05451 55828 
2.65196 13568 0.00097 17812 


n=6 

0.22284 66042 0.45896 46740 
1.18893 21017 0.4170008308 
2.99273 63261 0.1133733821 
5.77514 35691 0.01039 91975 
9.83746 74184 0.00026 10172 
15.9828 73981 0.00000 08985 


n=8 

0.38118 69902  0.6611470126 
1.1571937124 0.20780 23258 
1.98165 67567 0.01707 79830 
2.93063 74203 0.00019 96041 


n=7 

0.19304 36766 0.40931 89517 
1.02666 48953 0.42183 12779 
2.5678767450 0.14712 63487 
4.90035 30845 0.02063 35145 
8.18215 34446 0.00107 40101 
12.7341 80292 0.00001 58655 
19.395727862 0.00000 00317 


n=9 

0.0000000000 0.72023 52156 
0.7235510188 0.43265 15590 
1.46855 32892 0.08847 45274 
2.26658 05845 0.00494 36243 
3.19099 32018 0.00003 96070 


Table 27.9 The integration points and weights for a number of n-point 
Gauss—Laguerre and Gauss—Hermite integration formulae. Where the points 
are given as +x;, the contributions from both +x; and —x; must be included. 
However, the contribution from any point x; = 0 must be counted only once. 
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has become attached to methods based on randomly generated numbers — in 
many ways come into their own when used on multidimensional integrals over 
regions with complicated boundaries. 

It goes without saying that in order to use random numbers for calculational 
purposes a supply of them must be available. There was a time when they were 
provided in book form as a two-dimensional array of random digits in the range 
0 to 9. The user could generate the successive digits of a random number of 
any desired length by selecting their positions in the table in any predetermined 
and systematic way. Nowadays all computers and nearly all pocket calculators 
offer a function which supplies a sequence of decimal numbers, €, that, for all 
practical purposes, are randomly and uniformly chosen in the range 0 < € < 1. 
The maximum number of significant figures available in each random number 
depends on the precision of the generating device. We will defer the details of 
how these numbers are produced until later in this subsection, where it will 
also be shown how random numbers distributed in a prescribed way can be 
generated. 

All integrals of the general form shown in equation (27.34) can, by a suitable 
change of variable, be brought to the form 


é= [ f(x) dx, (27.48) 


and we will use this as our standard model. 

All approaches to integral evaluation based on random numbers proceed by 
estimating a quantity whose expectation value is equal to the sought-for value 0. 
The estimator t must be unbiased, i.e. we must have E[t] = 0, and the method 
must provide some measure of the likely error in the result. The latter will appear 
generally as the variance of the estimate, with its usual statistical interpretation, 
and not as a band in which the true answer is known to lie with certainty. 

The various approaches really differ from each other only in the degree of 
sophistication employed to keep the variance of the estimate of 0 small. The 
overall efficiency of any particular method has to take into account not only the 
variance of the estimate but also the computing and book-keeping effort required 
to achieve it. 

We do not have the space to describe even the most elementary methods in full 
detail, but the main thrust of each approach should be apparent to the reader 
from the brief descriptions that follow. 


Crude Monte Carlo 


The most straightforward application is one in which the random numbers are 
used to pick sample points at which f(x) is evaluated. These values are then 
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averaged: 
I, 
eee DSi (27.49) 


Stratified sampling 


Here the range of x is broken up into k subranges, 
0= a < % <s<m = 1, 


and crude Monte Carlo evaluation is carried out in each subrange. The estimate 
E[t] is then calculated as 


nj 


E{t] =D me INE (any + Eij(atj — 4-1) (27.50) 


fel igk nj 


This is an unbiased estimator of 0 with variance 


Pee ee ft ae a) , 
=>) se f [f(x)P dx — So — / jos 
j=l nj Oj j=l nj Oj 
This variance can be made less than that for crude Monte Carlo, whilst using 
the same total number of random numbers, n = 5° nj, if the differences between 
the average values of f(x) in the various subranges are significantly greater than 
the variations in f within each subrange. It is easier administratively to make all 
subranges equal in length, but better, if it can be managed, to make them such 
that the variations in f are approximately equal in all the individual subranges. 


Importance sampling 


Although we cannot integrate f(x) analytically — we would not be using Monte 
Carlo methods if we could — if we can find another function g(x) that can be 
integrated analytically and mimics the shape of f then the variance in the estimate 
of 0 can be reduced significantly compared with that resulting from the use of 
crude Monte Carlo evaluation. 

Firstly, if necessary the function g must be renormalised, so that G(x) 
So g(y)dy has the property G(1) = 1. Clearly, it also has the property G(0) = 


Then, since 
0= [2 F(x) G(s) 


it follows that finding the expectation value of f(y)/g(y) using a random number 
ny, distributed in such a way that € = G(n) is uniformly distributed on (0, 1), is 
equivalent to estimating 0. This involves being able to find the inverse function 
of G; a discussion of how to do this is given towards the end of this subsection. 
If g(7) mimics f(7) well, f(4)/g(y) will be nearly constant and the estimation 


0. 
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will have a very small variance. Further, any error in inverting the relationship 
between 7 and € will not be important since f(7)/g(y) will be largely independent 
of the value of 7. 

As an example, consider the function f(x) = [tan~!(x)]'/*, which is not analyt- 
ically integrable over the range (0, 1) but is well mimicked by the easily integrated 
function g(x) = x!/*(1 — x?/6). The ratio of the two varies from 1.00 to 1.06 as x 
varies from 0 to 1. The integral of g over this range is 0.619 048, and so it has to 
be renormalised by the factor 1.61538. The value of the integral of f(x) from 0 
to 1 can then be estimated by averaging the value of 


[tan“!(n)]'/ 
(1.615 38) 4!/2(1 — 112) 


for random variables 4 which are such that G(y) is uniformly distributed on 
(0, 1). Using batches of as few as ten random numbers gave a value 0.630 for 0, 
with standard deviation 0.003. The corresponding result for crude Monte Carlo, 
using the same random numbers, was 0.634 + 0.065. The increase in precision is 
obvious, though the additional labour involved would not be justified for a single 
application. 


Control variates 


The control-variate method is similar to, but not the same as, importance sam- 
pling. Again, an analytically integrable function that mimics f(x) in shape has 
to be found. The function, known as the control variate, is first scaled so as to 
match f as closely as possible in magnitude and then its integral is found in 
closed form. If we denote the scaled control variate by h(x), then the estimate of 
0 is computed as 


1 
t= [ve — h(x)] dx +f h(x) dx. (27.51) 


The first integral in (27.51) is evaluated using (crude) Monte Carlo, whilst the 
second is known analytically. Although the first integral should have been ren- 
dered small by the choice of h(x), it is its variance that matters. The method relies 
on the following result (see equation (30.136)): 


V({t—¢] =V[4q + VU[e'] — 2 Covie, t’], 


and on the fact that if ¢ estimates 0 whilst t’ estimates 0’ using the same random 
numbers, then the covariance of t and t’ can be larger than the variance of 1’, 
and indeed will be so if the integrands producing 0 and 6” are highly correlated. 

To evaluate the same integral as was estimated previously using importance 
sampling, we take as h(x) the function g(x) used there, before it was renormalised. 
Again using batches of ten random numbers, the estimated value for 0 was found 
to be 0.629 + 0.004, a result almost identical to that obtained using importance 
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sampling, in both value and precision. Since we knew already that f(x) and g(x) 
diverge monotonically by about 6% as x varies over the range (0,1), we could 
have made a small improvement to our control variate by scaling it by 1.03 before 
using it in equation (27.51). 


Antithetic variates 


As a final example of a method that improves on crude Monte Carlo, and one that 
is particularly useful when monotonic functions are to be integrated, we mention 
the use of antithetic variates. This method relies on finding two estimates t and 
t’ of @ that are strongly anticorrelated (i.e. Cov[t,t’] is large and negative) and 
using the result 


Vi5(t+¢)] = FV [4 + ZV Ie] + $Covfe, 1]. 


For example, the use of sf(O) + f(1 — €)] instead of f(€) involves only twice 
as many evaluations of f, and no more random variables, but generally gives 
an improvement in precision significantly greater than this. For the integral of 
f(x) = [tan-!(x)]'/?, using as previously a batch of ten random variables, an 
estimate of 0.623 + 0.018 was found. This is to be compared with the crude 
Monte Carlo result, 0.634 + 0.065, obtained using the same number of random 
variables. 

For a fuller discussion of these methods, and of theoretical estimates of their 
efficiencies, the reader is referred to more specialist treatments. For practical imple- 
mentation schemes, a book dedicated to scientific computing should be consulted 


Hit or miss method 


We now come to the approach that, in spirit, is closest to the activities that gave 
Monte Carlo methods their name. In this approach, one or more straightforward 
yes/no decisions are made on the basis of numbers drawn at random — the end 
result of each trial is either a hit or a miss! In this section we are concerned 
with numerical integration, but the general Monte Carlo approach, in which 
one estimates a physical quantity that is hard or impossible to calculate directly 
by simulating the physical processes that determine it, is widespread in modern 
science. For example, the calculation of the efficiencies of detector arrays in 
experiments to study elementary particle interactions are nearly always carried 
out in this way. Indeed, in a normal experiment, far more simulated interactions 
are generated in computers than ever actually occur when the experiment is 
taking real data. 

As was noted in chapter 2, the process of evaluating a one-dimensional integral 
r? f(x)dx can be regarded as that of finding the area between the curve y = f(x) 


S eg. W. H. Press, S. A. Teukolsky, W. T. Vetterling and B. P. Flannery, Numerical Recipes in C: The 
Art of Scientific Computing, 2nd edn (Cambridge: Cambridge University Press, 1992). 
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y=fx) 


bays x=b 


Figure 27.5 A simple rectangular figure enclosing the area (shown shaded) 
which is equal to f f (x) dx. 


and the x-axis in the range a < x < b. It may not be possible to do this 
analytically, but if, as shown in figure 27.5, we can enclose the curve in a simple 
figure whose area can be found trivially then the ratio of the required (shaded) 
area to that of the bounding figure, c(b — a), is the same as the probability that a 
randomly selected point inside the boundary will lie below the line. 

In order to accommodate cases in which f(x) can be negative in part of the 
x-range, we treat a slightly more general case. Suppose that, for a< x < b, f(x) 
is bounded and known to lie in the range A < f(x) < B; then the transformation 

x-—a 


b—a 


Z= 


will reduce the integral [ - f(x) dx to the form 


1 
Alba) +(B— ANb—a) f hia) dz, (27.52) 
0 


where 
aot al 
- BA 
In this form z lies in the range 0 < z < 1 and h(z) lies in the range 0 < A(z) < 1, 
i.e. both are suitable for simulation using the standard random-number generator. 
It should be noted that, for an efficient estimation, the bounds A and B should 
be drawn as tightly as possible —preferably, but not necessarily, they should be 
equal to the minimum and maximum values of f in the range. The reason for 
this is that random numbers corresponding to values which f(x) cannot reach 
add nothing to the estimation but do increase its variance. 

It only remains to estimate the final integral on the RHS of equation (27.52). 
This we do by selecting pairs of random numbers, €; and €), and testing whether 


h(z) [f ((b—a)z + a)— A]. 
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h(€,) > 2. The fraction of times that this inequality is satisfied estimates the value 
of the integral (without the scaling factors (B — A)(b — a)) since the expectation 
value of this fraction is the ratio of the area below the curve y = h(z) to the area 
of a unit square. 

To illustrate the evaluation of multiple integrals using Monte Carlo techniques, 
consider the relatively elementary problem of finding the volume of an irregular 
solid bounded by planes, say an octahedron. In order to keep the description 
brief, but at the same time illustrate the general principles involved, let us suppose 
that the octahedron has two vertices on each of the three Cartesian axes, one on 
either side of the origin for each axis. Denote those on the x-axis by x1(< 0) and 
x2(> 0), and similarly for the y- and z-axes. Then the whole of the octahedron 
can be enclosed by the rectangular parallelepiped 


XySXSx, WSVSy, 1757252. 


Any point in the octahedron lies inside or on the parallelepiped, but any point 
in the parallelepiped may or may not lie inside the octahedron. 

The equation of the plane containing the three vertex points (x;, 0,0), (0, y;,0) 
and (0,0, z,) is 


oe ee St for i, j,k = 1,2, (27.53) 


and the condition that any general point (x,y,z) lies on the same side of the 
plane as the origin is that 


ee Drees 12 (27.54) 
Xi Vj Zk 
For the point to be inside or on the octahedron, equation (27.54) must therefore 
be satisfied for all eight of the sets of i, j and k given in (27.53). 

Thus an estimate of the volume of the octahedron can be made by generating 
random numbers € from the usual uniform distribution and then using them in 
sets of three, according to the following scheme. 

With integer m labelling the mth set of three random numbers, calculate 


X = X1 + €3m—2(X2 — X1), 
y= t S3m-1(¥2 — V1)s 
z= 21 + €3m(z2 — 21). 
Define a variable n,, as 1 if (27.54) is satisfied for all eight combinations of i, j,k 


values and as 0 otherwise. The volume V can then be estimated using 3M random 
numbers from the formula 


M 
V 1 yy fm. 


@2—x)02—-yW@—21) M mal 
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It will be seen that, by replacing each n,, in the summation by f(x, y,z)mMm, 
this procedure could be extended to estimate the integral of the function f over 
the volume of the solid. The method has special valueif f is too complicated to 
have analytic integrals with respect to x,y and z or if the limits of any of these 
integrals are determined by anything other than the simplest combinations of the 
other variables. If large values of f are known to be concentrated in particular 
regions of the integration volume, then some form of stratified sampling should 
be used. 

It will be apparent that this general method can be extended to integrals 
of general functions, bounded but not necessarily continuous, over volumes with 
complicated bounding surfaces and, if appropriate, in more than three dimensions. 


Random number generation 


Earlier in this subsection we showed how to evaluate integrals using sequences of 
numbers that we took to be distributed uniformly on the interval 0 < € < 1. In 
reality the sequence of numbers is not truly random, since each is generated in 
a mechanistic way from its predecessor and eventually the sequence will repeat 
itself. However, the cycle is so long that in practice this is unlikely to be a problem, 
and the reproducibility of the sequence can even be turned to advantage when 
checking the accuracy of the rest of a calculational program. Much research has 
gone into the best ways to produce such ‘pseudo-random’ sequences of numbers. 
We do not have space to pursue them here and will limit ourselves to one recipe 
that works well in practice. 

Given any particular starting (integer) value xo, the following algorithm will 
generate a full cycle of m values for &;, uniformly distributed on 0 < €; < 1, 
before repeats appear: 


xj =axi-1+c (mod m); §=-—. 


Here c is an odd integer and a has the form a = 4k + 1, with k an integer. For 
practical reasons, in computers and calculators m is taken as a (fairly high) power 
of 2, typically the 32nd power. 

The uniform distribution can be used to generate random numbers y distributed 
according to a more general probability distribution f(y) on the rangea<y <b 
if the inverse of the indefinite integral of f can be found, either analytically or by 
means of a look-up table. In other words, if 


Ly 
FQ) = f, f(t)dt, 


for which F(a) = 0 and F(b) = 1, then F(y) is uniformly distributed on (0, 1). 
This approach is not limited to finite a and b; a could be —o and b could be o. 
The procedure is thus to select a random number € from a uniform distribution 
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on (0,1) and then take as the random number y the value of F~'(€). We now 
illustrate this with a worked example. 


> Find an explicit formula that will generate a random number y distributed on (—o0, 0) 
according to the Cauchy distribution 


fordy = (2) sos, 


given a random number € uniformly distributed on (0, 1). 


The first task is to determine the indefinite integral: 


Yra dt {i ay 1 
rov= | (2) = -—tan pears 


ti/@+? 7 


Now, if y is distributed as we wish then F(y) is uniformly distributed on (0, 1). This follows 
from the fact that the derivative of F(y) is f(y). We therefore set F(y) equal to € and 
obtain 


1 
€=—tant 24+ 
T a 


yielding 
y =atan[n(é — 5)]. 
This explicit formula shows how to change a random number € drawn from a population 


uniformly distributed on (0,1) into a random number y distributed according to the 
Cauchy distribution. < 


Look-up tables operate as described below for cumulative distributions F(y) 
that are non-invertible, ic. F~'(y) cannot be expressed in closed form. They 
are especially useful if many random numbers are needed but great sampling 
accuracy is not essential. The method for an N-entry table can be summarised as 
follows. Define wn by F(wm) = m/N for m = 1,2,...,N, and store a table of 


y(m) = 3(Wm + Wm-1)- 
As each random number y is needed, calculate k as the integral part of Né and 
take y as given by y(k). 

Normally, such a look-up table would have to be used for generating random 
numbers with a Gaussian distribution, as the cumulative integral of a Gaussian is 
non-invertible. It would be, in essence, table 30.3, with the roles of argument and 
value interchanged. In this particular case, an alternative, based on the central 
limit theorem, can be considered. 

With € generated in the usual way, i.e. uniformly distributed on the interval 
0 < é <1, the random variable 


n 
y= &-jn (27.55) 
i=1 


is normally distributed with mean 0 and variance n/12 when n is large. This 
approach does produce a continuous spectrum of possible values for y, but needs 
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many values of &; for each value of y and is a very poor approximation if the 
wings of the Gaussian distribution have to be sampled accurately. For nearly all 
practical purposes a Gaussian look-up table is to be preferred. 


27.5 Finite differences 


It will have been noticed that earlier sections included several equations linking 
sequential values of f; and the derivatives of f evaluated at one of the x;. In 
this section, by way of preparation for the numerical treatment of differential 
equations, we establish these relationships in a more systematic way. 

Again we consider a set of values f; of a function f(x) evaluated at equally 
spaced points x;, their separation being h. As before, the basis for our discussion 
will be a Taylor series expansion, but on this occasion about the point x;: 


: : , Ww, i 
fir = fit hf; 4 malt 7 
In this section, and subsequently, we denote the nth derivative evaluated at x; 
by fi”. 

From (27.56), three different expressions that approximate q? can be derived. 
The first of these, obtained by subtracting the + equations, is 


: df fi —fin W, 
(00 ee (cn ae cic A ng ea) A 
f= (4) 2h afi Cant) 


far, (27.56) 


The quantity (fi+1 — fi-1)/(2h) 1s known as the central difference approximation 
to fig and can be seen from (27.57) to be in error by approximately (h2/6)f°>. 
An alternative approximation, obtained from (27.56+) alone, is given by 


wf) - faiah . £6 
i =(Z), h ile ee (27.58) 


The forward difference approximation, (f:+; — f;)/h, is clearly a poorer approxi- 
mation, since it is in error by approximately (h/2)f as compared with (h2/6)f. 
Similarly, the backward difference (f; — fi-1)/h obtained from (27.56—) is not as 
good as the central difference; the sign of the error is reversed in this case. 
This type of differencing approximation can be continued to the higher deriva- 
tives of f in an obvious manner. By adding the two equations (27.56+), a central 
gs -(2) eee 
difference approximation to f;” can be obtained: 


2 7 ~ 
2 (4h) _ fier —2fi +fi- 
f= (3) a Hn (27.59) 


The error in this approximation (also known as the second difference of f) is 
easily shown to be about (h2/12)f. 
Of course, if the function f(x) is a sufficiently simple polynomial in x, all 
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derivatives beyond a particular one will vanish and there is no error in taking 
the differences to obtain the derivatives. 


> The following is copied from the tabulation of a second-degree polynomial f(x) at values 
of x from 1 to 12 inclusive: 


Dy Poy Uo, 13. Ms PINS BPD, NGS, ls GL, OY, II, 


The entries marked ? were illegible and in addition one error was made in transcription. 
Complete and correct the table. Would your procedure have worked if the copying error 
had been in f(6)? 


Write out the entries again in row (a) below, and where possible calculate first differences 
in row (b) and second differences in row (c). Denote the jth entry in row (n) by (n)j. 


(a) 2 2 ? 8 14 22 32 46 ? 74 92 112 
(b) 0 ? ? 6 8 10 14 ? ? 18 20 
(c) 2 2 2 2 2 4 ? 9 9 2 


Because the polynomial is second-degree, the second differences (c);, which are proportional 
to d’f/dx?, should be constant, and clearly the constant should be 2. That is, (c)g should 
equal 2 and (b)7 should equal 12 (not 14). Since all the (c); = 2, we can conclude that 
(b)2 = 2, (b)3 = 4, (b)g = 14, and (b)y = 16. Working these changes back to row (a) shows 
that (a); = 4, (a)g = 44 (not 46), and (a)y = 58. 

The entries therefore should read 


(a) 2, 2, 4, 8, 14, 22, 32, 44, 58, 74, 92, 112, 


where the amended entries are shown in bold type. 

It is easily verified that if the error were in f(6) no two computable entries in row (c) 
would be equal, and it would not be clear what the correct common entry should be. 
Nevertheless, trial and error might arrive at a self-consistent scheme. < 


27.6 Differential equations 


For the remaining sections of this chapter our attention will be on the solution 
of differential equations by numerical methods. Some of the general difficulties 
of applying numerical methods to differential equations will be all too apparent. 
Initially we consider only the simplest kind of equation — one of first order, 
typically represented by 


d 
= = fy) (27.60) 


where y is taken as the dependent variable and x the independent one. If this 
equation can be solved analytically then that is the best course to adopt. But 
sometimes it is not possible to do so and a numerical approach becomes the 
only one available. In fact, most of the examples that we will use can be solved 
easily by an explicit integration, but, for the purposes of illustration, this is an 
advantage rather than the reverse since useful comparisons can then be made 
between the numerically derived solution and the exact one. 
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y(exact) 


mM dQ W (1) (1) 
0.590 0.500 —0.500 0.607 
0.349 0.250 0.250 0.368 
0.206 0.125 —0.125 0.223 
0.122 0.063 0.063 0.135 
0.072 0.032 —0.032 0.082 
0.042 0.016 0.016 0.050 


Table 27.10 The solution y of differential equation (27.61) using the Euler 
forward difference method for various values of h. The exact solution is also 
shown. 


27.6.1 Difference equations 
Consider the differential equation 


i me (1) (27.61) 
dx 

and the possibility of solving it numerically by approximating dy/dx by a finite 
difference along the lines indicated in section 27.5. We start with the forward 


difference 


dy Viti — Ji 
— | eos 27.62 
a). h ( ) 


where we use the notation of section 27.5 but with f replaced by y. In this 
particular case, it leads to the recurrence relation 


ie SHS (4) = yi hy = My 27.63) 
T 

Thus, since yo = y(0) = 1 is given, y, = y(0+h) = y(h) can be calculated, and 

so on (this is the Euler method). Table 27.10 shows the values of y(x) obtained 

if this is done using various values of h and for selected values of x. The exact 

solution, y(x) = exp(—x), is also shown. 

It is clear that to maintain anything like a reasonable accuracy only very small 
steps, h, can be used. Indeed, if h is taken to be too large, not only is the accuracy 
bad but, as can be seen, for h > 1 the calculated solution oscillates (when it 
should be monotonic), and for h > 2 it diverges. Equation (27.63) is of the form 
Yitt = Ay;, and a necessary condition for non-divergence is |A| < 1, ie. 0<h <2, 
though in no way does this ensure accuracy. 

Part of this difficulty arises from the poor approximation (27.62); its right- 
hand side is a closer approximation to dy/dx evaluated at x = x; +h/2 than to 
dy/dx at x = x;. This is the result of using a forward difference rather than the 
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x y(estim.) — y(exact) 


—05 (1.648) — 
0 (1.000) (1.000) 
0.5 0.648 0.607 
1.0 0.352 0.368 
L5 0.296 0.223 
2.0 0.056 0.135 


2.5 0.240 0.082 
3.0 —0.184 0.050 


Table 27.11 The solution of differential equation (27.61) using the Milne 
central difference method with h = 0.5 and accurate starting values. 


more accurate, but of course still approximate, central difference. A more accurate 
method based on central differences (Milne’s method) gives the recurrence relation 


d 
you = yea +2 (<) (27.64) 


i 


in general and, in this particular case, 
Vint = Vit — 2hyj. (27.65) 


An additional difficulty now arises, since two initial values of y are needed. 
The second must be estimated by other means (e.g. by using a Taylor series, 
as discussed later), but for illustration purposes we will take the accurate value, 
y(—h) = exp h, as the value of y_1. If h is taken as, say, 0.5 and (27.65) is applied 
repeatedly, then the results shown in table 27.11 are obtained. 

Although some improvement in the early values of the calculated y(x) is 
noticeable, as compared with the corresponding (h = 0.5) column of table 27.10, 
this scheme soon runs into difficulties, as is obvious from the last two rows of the 
table. 

Some part of this poor performance is not really attributable to the approxi- 
mations made in estimating dy/dx but to the form of the equation itself and 
hence of its solution. Any rounding error occurring in the evaluation effectively 
introduces into y some contamination by the solution of 


This equation has the solution y(x) = exp x and so grows without limit; ultimately 
it will dominate the sought-for solution and thus render the calculations totally 
inaccurate. 

We have only illustrated, rather than analysed, some of the difficulties associated 
with simple finite-difference iteration schemes for first-order differential equations, 


1022 


27.6 DIFFERENTIAL EQUATIONS 


but they may be summarised as (1) insufficiently precise approximations to the 
derivatives and (ii) inherent instability due to rounding errors. 


27.6.2 Taylor series solutions 


Since a Taylor series expansion is exact if all its terms are included, and the limits 
of convergence are not exceeded, we may seek to use one to evaluate y1, yz, etc. 
for an equation 

dy : 

= =f (x,y), (27.66) 


dx 


when the initial value y(xo) = yo is given. 
The Taylor series is 


I h3 
y(x +h) = y(x) + hy’) + yy") + FMC +--- (27.67) 
In the present notation, at the point x = x; this is written 
er ALIN eee) 
Yin =Yithyy + ai a 31 Teas (27.68) 
But, for the required solution y(x), we know that 
d 
y= () = fn. (27.69) 
XJ, 
and the value of the second derivative at x = x;, y = y; can be obtained from it: 
ge seb eae’ a 
2 TO 2 Fo (27.70) 


dx | dydx dx” dy’ 
This process can be continued for the third and higher derivatives, all of which 
are to be evaluated at (xj, yj). 


Having obtained expressions for the derivatives yo” in (27.67), two alternative 
ways of proceeding are open to us: 


(i) equation (27.68) is used to evaluate y;,1, the whole process is repeated to 
obtain yi;2, and so on; 

(ii) equation (27.68) is applied several times but using a different value of h 
each time, and so the corresponding values of y(x +h) are obtained. 


It is clear that, on the one hand, approach (i) does not require so many terms of 
(27.67) to be kept, but, on the other hand, the y;(m) have to be recalculated at 
each step. With approach (ii), fairly accurate results for y may be obtained for 
values of x close to the given starting value, but for large values of h a large 
number of terms of (27.67) must be kept. As an example of approach (ii) we 
solve the following problem. 
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x y(estim.) —y(exact) 
i) 1.0000 1.0000 
0.1 = 1.2346 1.2346 
0.2 1.5619 1.5625 
0.3. 2.0331 2.0408 
0.4 2.7254 2.7778 
0.5 3.7500 4.0000 


Table 27.12 The solution of differential equation (27.71) using a Taylor series. 


> Find the numerical solution of the equation 
dy 


ae Oe (27.71) 


for x =0.1 to 0.5 in steps of 0.1. Compare it with the exact solution obtained analytically. 


Since the right-hand side of the equation does not contain x explicitly, (27.70) is greatly 
simplified and the calculation becomes a repeated application of 


(n+l) _ oy”) dy as ay” 


; dy dx oy” 
The necessary derivatives and their values at x = 0, where y = 1, are given below: 
y(0)=1 1 
yl = 2y3? 2 
y! = (3/2)(2y!?)2y*?) = 6y? 6 
y® = (12y)2y3/? = 24y°? 24 


y = (60y9/7)2y3? = 120y3 120 
yS) = (360y?)2y3/? = 720y7/?, 720 

Thus the Taylor expansion of the solution about the origin (in fact a Maclaurin series) is 
24 120 720 


6 
W(x) = 1+ 2x + 593? + Spx) + exe + Se to 


Hence, y(estim.) = 1 + 2x + 3x? + 4x3 + 5x4 + 6x°. Values calculated from this are given 
in table 27.12. Comparison with the exact values shows that using the first six terms gives 
a value that is correct to one part in 100, up to x = 0.3. < 


27.6.3 Prediction and correction 


An improvement in the accuracy obtainable using difference methods is possible 
if steps are taken, sometimes retrospectively, to allow for inaccuracies in approx- 
imating derivatives by differences. We will describe only the simplest schemes of 
this kind and begin with a prediction method, usually called the Adams method. 
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The forward difference estimate of yj+1, namely 


d . 
vn = wth (2) = hfs (27.72 
i 
would give exact results if y were a linear function of x in the range x; < x < x;+h. 
The idea behind the Adams method is to allow some relaxation of this and 
suppose that y can be adequately approximated by a parabola over the interval 
Xi-1 < xX < Xi41. In the same interval, dy/dx can then be approximated by a linear 
function: 
_ dy 
f(x, y) = dx 
The values of a and b are fixed by the calculated values of f at x;_; and x;, which 
we may denote by fj_; and f;: 


wzatb(x—x;) forx;-h<x<x,+h. 


b fi-fi-n 


h 


a=fi, 
Thus 


xith pine 4 
Viti — Vi ® i, fi + Feu. — xi)| dx, 


which yields 
yer = i thf; + Zh(fi — fi-t). (27.73) 


The last term of this expression is seen to be a correction to result (27.72). That 
it is, in some sense, the second-order correction, 

Sy?) 195 
to a first-order formula is apparent. 

Such a procedure requires, in addition to a value for yo, a value for either y, or 
y_1, so that f; or f_; can be used to initiate the iteration. This has to be obtained 
by other methods, e.g. a Taylor series expansion. 

Improvements to simple difference formulae can also be obtained by using 
correction methods. In these, a rough prediction of the value yj;1 is made first, 
and then this is used in a better formula, not originally usable since it, in turn, 
requires a value of yj41 for its evaluation. The value of yj41 is then recalculated, 
using this better formula. 

Such a scheme based on the forward difference formula might be as follows: 


(i) predict yj+1 using yir1 = yi +hfi; 
(ii) calculate fi,; using this value; 
(ii) recalculate yiy, using yipr = yi + h(fi + fizs)/2. Here (fi + fiz1)/2 has 
replaced the f; used in (i), since it better represents the average value of 
dy/dx in the interval x} <x <x; +h. 
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Steps (ii) and (ii) can be iterated to improve further the approximation to the 
average value of dy/dx, but this will not compensate for the omission of higher- 
order derivatives in the forward difference formula. 

Many more complex schemes of prediction and correction, in most cases 
combining the two in the same process, have been devised, but the reader is 
referred to more specialist texts for discussions of them. However, because it 
offers some clear advantages, one group of methods will be set out explicitly in 
the next subsection. This is the general class of schemes known as Runge-Kutta 
methods. 


27.6.4 Runge-Kutta methods 


The Runge-Kutta method of integrating 
d : 
& = f(x.9) (27.74) 


is a step-by-step process of obtaining an approximation for y;; by starting from 
the value of y;. Among its advantages are that no functions other than f are used, 
no subsidiary differentiation is needed and no additional starting values need be 
calculated. 

To be set against these advantages is the fact that f is evaluated using somewhat 
complicated arguments and that this has to be done several times for each increase 
in the value of i. However, once a procedure has been established, for example 
on a computer, the method usually gives good results. 

The basis of the method is to simulate the (accurate) Taylor series for y(x; +A), 
not by calculating all the higher derivatives of y at the point x; but by taking 
a particular combination of the values of the first derivative of y evaluated at 
a number of carefully chosen points. Equation (27.74) is used to evaluate these 
derivatives. The accuracy can be made to be up to whatever power of h is desired, 
but, naturally, the greater the accuracy, the more complex the calculation, and, 
in any case, rounding errors cannot ultimately be avoided. 

The setting up of the calculational scheme may be illustrated by considering 
the particular case in which second-order accuracy in h is required. To second 
order, the Taylor expansion is 


_ We (df 
Vier = yi thfi ul ; (27.75) 
2 \dx/,, 


where 
df\ fof , ,of\ _ of; (fi 
(S) (E+) te tap 


the last step being merely the definition of an abbreviated notation. 
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We assume that this can be simulated by a form 


Yin = Yet aahfi t+ oahf (xi + Bih, yi + Bohfi), (27.76) 


which in effect uses a weighted mean of the value of dy/dx at x; and its value at 
some point yet to be determined. The object is to choose values of a1, #2, 6; and 
By such that (27.76) coincides with (27.75) up to the coefficient of h?. 

Expanding the function f in the last term of (27.76) in a Taylor series of its 
own, we obtain 


FGF Bik, yi + Bohfi) = FOG vi) + Byrd + Bahfi 


Ox 


oh 4 O(n), 
ey 


Putting this result into (27.76) and rearranging in powers of h, we obtain 


yout = yet (on + o)hfs + oh? (az + bof ) (27.77) 


Ox oy 


Comparing this with (27.75) shows that there is, in fact, some freedom remaining 
in the choice of the «’s and fs. In terms of an arbitrary a, (# 1), 


1 


oF) on, Bi = Bo ia) 


One possible choice is « = 0.5, giving #7 = 0.5, 6; = By = 1. In this case the 
procedure (equation (27.76)) can be summarised by 


Viet = Vi t+ $(a1 + a), (27.78) 
where 
a, = hf (xi, yi), 
ay = hf(xi +h, y;+ a1). 


Similar schemes giving higher-order accuracy in h can be devised. Two such 
schemes, given without derivation, are as follows. 


(i) To order h3, 


Vier = yi t 2(b1 + 4b2 + bs), (27.79) 
where 
by = hf (xi. yi), 
by = hf (x; + 5h, yi t $b1), 
b3 =hf(x; +h, yi + 2b. — by). 
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(ii) To order h4, 
Viti = Vit z(c1 + 2cz + 2c3 + c4), (27.80) 
where 
c1 = hf (xi, yi), 
co =hf(xit sh, vit 5¢1), 
c3 = hf (xi + sh, yi + 502), 
cq = hf(xj +h, yi +3). 


27.6.5 Isoclines 


The final method to be described for first-order differential equations is not so 
much numerical as graphical, but since it is sometimes useful it is included here. 
The method, known as that of isoclines, involves sketching for a number of 
values of a parameter c those curves (the isoclines) in the xy-plane along which 
f(x,y) =, Le. those curves along which dy/dx is a constant of known value. It 
should be noted that isoclines are not generally straight lines. Since a straight 
line of slope dy/dx at and through any particular point is a tangent to the curve 
y = y(x) at that point, small elements of straight lines, with slopes appropriate 
to the isoclines they cut, effectively form the curve y = y(x). 
Figure 27.6 illustrates in outline the method as applied to the solution of 


Oo 2. Diy (27.81) 


The thinner curves (rectangular hyperbolae) are a selection of the isoclines along 
which —2xy is constant and equal to the corresponding value of c. The small 
cross lines on each curve show the slopes (= c) that solutions of (27.81) must 
have if they cross the curve. The thick line is the solution for which y = 1 at 
x = 0; it takes the slope dictated by the value of c on each isocline it crosses. The 
analytic solution with these properties is y(x) = exp(—x?). 


27.7 Higher-order equations 


So far the discussion of numerical solutions of differential equations has been 
in terms of one dependent and one independent variable related by a first-order 
equation. It is straightforward to carry out an extension to the case of several 
dependent variables y;,; governed by R first-order equations: 


dyin 
dx 
We have enclosed the label r in brackets so that there is no confusion between, 
say, the second dependent variable yj) and the value y, of a variable y at the 


= f iq pay Ypp---> Yeap r=1,2,...,R. 
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Figure 27.6 The isocline method. The cross lines on each isocline show the 


slopes that solutions of dy/dx = —2xy must have at the points where they 
cross the isoclines. The heavy line is the solution with y(0) = 1, namely 
exp(—x?). 


second calculational point x2. The integration of these equations by the methods 
discussed in the previous section presents no particular difficulty, provided that 
all the equations are advanced through each particular step before any of them 
is taken through the following step. 

Higher-order equations in one dependent and one independent variable can be 
reduced to a set of simultaneous equations, provided that they can be written in 
the form 


Sk = FYI YO), (27.82) 


where R is the order of the equation. To do this, a new set of variables pj, is 
defined by 


d'y 


ar THhB ROL (27.83) 


Pfr] 4 


Equation (27.82) is then equivalent to the following set of simultaneous first-order 
equations: 


dy 
as Pit}, 
d 
Pel pin, r= 1,2,...,R—2, (27.84) 
dx 
dpir— : 
ns = f(x,y, py... PtR-t))- 


1029 


NUMERICAL METHODS 


These can then be treated in the way indicated in the previous paragraph. The 
extension to more than one dependent variable is straightforward. 

In practical problems it often happens that boundary conditions applicable 
to a higher-order equation consist not of the values of the function and all its 
derivatives at one particular point but of, say, the values of the function at two 
separate end-points. In these cases a solution cannot be found using an explicit 
step-by-step ‘marching’ scheme, in which the solutions at successive values of the 
independent variable are calculated using solution values previously found. Other 
methods have to be tried. 

One obvious method is to treat the problem as a ‘marching one’, but to use a 
number of (intelligently guessed) initial values for the derivatives at the starting 
point. The aim is then to find, by interpolation or some other form of iteration, 
those starting values for the derivatives that will produce the given value of the 
function at the finishing point. 

In some cases the problem can be reduced by a differencing scheme to a matrix 
equation. Such a case is that of a second-order equation for y(x) with constant 
coefficients and given values of y at the two end-points. Consider the second-order 
equation 


y" + 2ky! + uy = f(x), (27.85) 
with the boundary conditions 
yO)=A4, yd) = B. 


If (27.85) is replaced by a central difference equation, 


Vig — 2yi + Vi-n Vint — 
2k 
h t 2h 
we obtain from it the recurrence relation 
(1+ kh)yigs + (uh? — 2)yi + 1 — kh) ya = If (x)). 


For h = 1/(N — 1) this is in exactly the form of the N x N tridiagonal matrix 
equation (27.30), with 


aN Lyi = f (xi), 


b; = by = 1, cy = ay = 0, 
a; = 1—kh, b; = ph? —2, c¢; =1+kh, i= 2,3,...,.N—1, 


and y, replaced by A, yy by B and y; by h?f(x;) for i = 2,3,...,N —1. The 
solutions can be obtained as in (27.31) and (27.32). 


27.8 Partial differential equations 


The extension of previous methods to partial differential equations, thus involving 
two or more independent variables, proceeds in a more or less obvious way. Rather 
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than an interval divided into equal steps by the points at which solutions to the 
equations are to be found, a mesh of points in two or more dimensions has to be 
set up and all the variables given an increased number of subscripts. 

Considerations of the stability, accuracy and feasibility of particular calcula- 
tional schemes are the same as for the one-dimensional case in principle, but in 
practice are too complicated to be discussed here. 

Rather than note generalities that we are unable to pursue in any quantitative 
way, we will conclude this chapter by indicating in outline how two familiar 
partial differential equations of physical science can be set up for numerical 
solution. The first of these is Laplace’s equation in two dimensions, 

Op eh _ 


0x2 dy? 


0, (27.86) 


the value of # being given on the perimeter of a closed domain. 

A grid with spacings Ax and Ay in the two directions is first chosen, so that, 
for example, x; stands for the point x9 + iAx and i, for the value (xj, y;). Next, 
using a second central difference formula, (27.86) is turned into 

bittj — 261; + Gig . Gijti— 26.7 + Gij-1 
(Ax : 


=0, 
(Ay) (27.87) 


for i= 0,1,...,N and j = 0,1,...,M. If (Ax)? = A(Ay)? then this becomes the 
recurrence relationship 


ist y + bi-1y + Mbijzt + bij-1) = 20. + A)di;- (27.88) 


The boundary conditions in their simplest form (i.e. for a rectangular domain) 
mean that 


goj, nj, Gio, Pim (27.89) 


have predetermined values. Non-rectangular boundaries can be accommodated, 
either by more complex boundary-value prescriptions or by using non-Cartesian 
coordinates. 

To find a set of values satisfying (27.88), an initial guess of a complete set of 
values for the ¢;; is made, subject to the requirement that the quantities listed 
in (27.89) have the given fixed values; those values that are not on the boundary 
are then adjusted iteratively in order to try to bring about condition (27.88) 
everywhere. Clearly one scheme is to set 4 = 1 and recalculate each ¢;,; as the 
mean of the four current values at neighbouring grid-points, using (27.88) directly, 
and then to iterate this recalculation until no value of ¢ changes significantly 
after a complete cycle through all values of i and j. This procedure is the simplest 
of such ‘relaxation’ methods; for a slightly more sophisticated scheme see exercise 
27.26 at the end of this chapter. The reader is referred to specialist books for 
fuller accounts of how this approach can be made faster and more accurate. 
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Our final example is based upon the one-dimensional diffusion equation for 
the temperature ¢ of a system: 


dp &h 


— =k. 
ot Ox? 


(27.90) 


If $;,; stands for @(xo + iAx, to + jAt) then a forward difference representation 
of the time derivative and a central difference representation for the spatial 
derivative lead to the following relationship: 


bijti— bij — bit, — 2615 + Gi 
- K a (27.91) 


This allows the construction of an explicit scheme for generating the temperature 
distribution at later times, given that it is known at some earlier time: 


ifr = Upisiy + Pi-1j) + — 20 gi;, (27.92) 


where o = xAt/(Ax)?. 

Although this scheme is explicit, it is not a good one because of the asymmetric 
way in which the differences are formed. However, the effect of this can be 
minimised if we study and correct for the errors introduced in the following way. 
Taylor’s series for the time variable gives 


Ogig , (At)? Odi; 
Ot 2! at? : 


Pij+i = bij + At (27.93) 


using the same notation as previously. Thus the first correction term to the LHS 
of (27.91) is 


(27.94) 


The first term omitted on the RHS of the same equation is, by a similar argument, 


2(Ax)? 04 gi; 


a aa (27.95) 
But, using the fact that @ satisfies (27.90), we obtain 
eo 2 oo a (0h ,0+o 
—s Se (KS) HG | td) cere 27.96 
ae ot (« 5) “Ox (S) © Ox? (2728) 


and so, to this accuracy, the two errors (27.94) and (27.95) can be made to cancel 
if « is chosen such that 
«At = 2«(Ax)* 
2 4! 


pete 
; £4 =F. 
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27.1 


27.2 


27.3 


27.4 


27.5 


27.6 


27.7 


27.9 Exercises 


Use an iteration procedure to find the root of the equation 40x = expx to four 
significant figures. 

Using the Newton—Raphson procedure find, correct to three decimal places, the 
root nearest to 7 of the equation 4x* + 2x” — 200x — 50 = 0. 

Show the following results about rearrangement schemes for polynomial equa- 
tions. 


(a) That if a polynomial equation g(x) = x” — f(x) = 0, where f(x) is a 
polynomial of degree less than m and for which f(0) ¥ 0, is solved using 
a rearrangement iteration scheme x,4,; = [f(x,)]!/", then, in general, the 
scheme will have only first-order convergence. 
(b) By considering the cubic equation 
x? — ax? + 2abx — (b° + ab?) =0 


for arbitrary non-zero values of a and b, demonstrate that, in special cases, the 
same rearrangement scheme can give second- (or higher-) order convergence. 


The square root of a number N is to be determined by means of the iteration 
scheme 


Xn4i = Xn [1 — (N fA xa) f(N)] 7 
Determine how to choose f(N) so that the process has second-order convergence. 
Given that ,/7 ~ 2.65, calculate \/7 as accurately as a single application of the 
formula will allow. 
Solve the following set of simultaneous equations using Gaussian elimination 
(including interchange where it is formally desirable): 
Xi + 3x 7 4x3 a 2x4 = 0, 
2x1 + 10x, = 5x3 +X4 = 6, 
4x, = 3x3 = 3x4 = 20, 
—3x, + 6x2 + 12x; = 4x4 = 16, 


The following table of values of a polynomial p(x) of low degree contains an 
error. Identify and correct the erroneous value and extend the table up to x = 1.2. 


x p(x) xX pls) 

0.0 0.000 0.5 0.165 
0.1 0.011 0.6 0.216 
0.2 0.040 0.7 0.245 
0.3 0.081 0.8 0.256 
0.4 0.128 0.9 0.243 


Simultaneous linear equations that result in tridiagonal matrices can sometimes 
be treated as three-term recurrence relations, and their solution may be found 
in a similar manner to that described in chapter 15. Consider the tridiagonal 
simultaneous equations 


Xi + 4x; + X41 = 3(i410 — 6-10), 7=0,41,42,.... 


Prove that, for i > 0, the equations have a general solution of the form x; = 
ap! + Bq', where p and q are the roots of a certain quadratic equation. Show that 
a similar result holds for i < 0. In each case express Xo in terms of the arbitrary 
constants «, B,.... 

Now impose the condition that x; is bounded as i > +00 and obtain a unique 
solution. 
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27.8 


27.9 


27.10 


27.11 


27.12 


A possible rule for obtaining an approximation to an integral is the mid-point 
rule, given by 


xg +Ax 
/ f(x) dx = Ax f(xo + Ax) + O(Ax?). 
xo 

Writing h for Ax, and evaluating all derivates at the mid-point of the interval 
(x, x + Ax), use a Taylor series expansion to find, up to O(h>), the coefficients of 
the higher-order errors in both the trapezium and mid-point rules. Hence find a 
linear combination of these two rules that gives O(h>) accuracy for each step Ax. 
Although it can easily be shown, by direct calculation, that 


oo = 2 1 
[ e * cos(kx) dx = Toke’ 


the form of the integrand is appropriate for Gauss—Laguerre numerical integra- 
tion. Using a 5-point formula, investigate the range of values of k for which the 
formula gives accurate results. At about what value of k do the results become 
inaccurate at the 1% level? 

Using the points and weights given in table 27.9, answer the following questions. 


(a) A table of unnormalised Hermite polynomials H,,(x) has been spattered with 
ink blots and gives Hs(x) as 32x°—?x? + 120x and H4(x) as ?x*—?x? + 12, 
where the coefficients marked ? cannot be read. What should they read? 

(b) What is the value of the integral 


(oo) ex d 
I= ——— dx, 
[ » 4X7 +3x+4+1 i 
as given by a 7-point integration routine? 


Consider the integrals I, defined by 


1 x2? 
I, = i, ———— dx. 
. -1J1—~x? 
(a) By setting x = sin@ and using the results given in exercise 2.42, show that I, 
has the value 


(b) Evaluate I, for p = 1,2,... ,6 using 5- and 6-point Gauss—Chebyshev inte- 
gration (conveniently run on a spreadsheet such as Excel) and compare the 
results with those in (a). In particular, show that, as expected, the 5-point 
scheme first fails to be accurate when the order of the polynomial numerator 
(2p) exceeds (2 x 5)—1 = 9. Likewise, verify that the 6-point scheme evaluates 
I; accurately but is in error for I¢. 


In normal use only a single application of n-point Gaussian quadrature is made, 
using a value of n that is estimated from experience to be ‘safe’. However, it is 
instructive to examine what happens when n is changed in a controlled way. 


(a) Evaluate the integral 


5 
I, -| V7x — x? — 10dx 
2 


using n-point Gauss—Legendre formulae for n = 2,3,...,6. Estimate (to 4 
s.f.) the value I,, you would obtain for very large n and compare it with the 
result J obtained by exact integration. Explain why the variation of J, with 
n is monotonically decreasing. 
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27.13 


27.14 


27.15 


27.16 


27.17 


(b) Try to repeat the processes described in (a) for the integrals 


; 1 
oy -| ——_—_——_ dx. 
2 V7x—x?—10 


Why is it very difficult to estimate J,,? 


Given a random number y uniformly distributed on (0, 1), determine the function 
€ = (yn) that would generate a random number € distributed as 

(a) 2€¢ on O<€<1, 

(b) 3 € on 0<é<1, 


T né 
(c) ecoss~ on —ax<i<a, 
(d) dexp(—|é|) on -a<é <0. 


A, B and C are three circles of unit radius with centres in the xy-plane at 
(1, 2), (2.5, 1.5) and (2, 3), respectively. Devise a hit or miss Monte Carlo calculation 
to determine the size of the area that lies outside C but inside A and B, as well 
as inside the square centred on (2, 2.5), that has sides of length 2 parallel to the 
coordinate axes. You should choose your sampling region so as to make the 
estimation as efficient as possible. Take the random number distribution to be 
uniform on (0,1) and determine the inequalities that have to be tested using the 
random numbers chosen. 
Use a Taylor series to solve the equation 

& + xy =0, y(0) = 1, 

dx 
evaluating y(x) for x = 0.0 to 0.5 in steps of 0.1. 
Consider the application of the predictor—-corrector method described near the 
end of subsection 27.6.3 to the equation 

dy 

oe y(0) = 0. 
Show, by comparison with a Taylor series expansion, that the expression obtained 
for yii1 in terms of x; and y; by applying the three steps indicated (without any 
repeat of the last two) is correct to O(h’). Using steps of h = 0.1 compute the 
value of y(0.3) and compare it with the value obtained by solving the equation 
analytically. 
A more refined form of the Adams predictor—corrector method for solving the 
first-order differential equation 

dy 
Fe LOY) 

is known as the Adams—Moulton—Bashforth scheme. At any stage (say the nth) 
in an Nth-order scheme, the values of x and y at the previous N solution points 
are first used to predict the value of y,41. This approximate value of y at the 
next solution point, x,4,, denoted by y,41, is then used together with those at the 
previous N — 1 solution points to make a more refined (corrected) estimation of 
Y(Xn41). The calculational procedure for a third-order scheme is summarised by 
the two following two equations: 


Vn+t = Vn + h(afn + arf + a3fn-2) (predictor), 
Vnt+t = Yn A(bif (Xn41, Vast) + bofn an b3fi—-1) (corrector). 


(a) Find Taylor series expansions for f,_; and f,-2 in terms of the function 
fn =f (Xn, Yn) and its derivatives at x). 
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27.18 


27.19 


27.20 


(b) Substitute them into the predictor equation and, by making that expression 
for Yn41 coincide with the true Taylor series for y,41 up to order h}, establish 
simultaneous equations that determine the values of a;,a2 and a3. 

(c) Find the Taylor series for f,4; and substitute it and that for f,_; into the 
corrector equation. Make the corrected prediction for y,,, coincide with the 
true Taylor series by choosing the weights b;,b, and b3 appropriately. 

(d) The values of the numerical solution of the differential equation 

dy 21+x)y +x3/? 


ax 2x(1 +x) 
at three values of x are given in the following table: 


x 0.1 0.2 0.3 
y(x) 0.030628 0.084107 0.150328 


Use the above predictor—corrector scheme to find the value of y(0.4) and 
compare your answer with the accurate value, 0.225 577. 
If dy/dx = f(x,y) then show that 


Cf _ vf Pf | 20 , of of af)? 
dx? ant S axay 7S dy? +2 + e(S) : 


Hence verify, by substitution and the subsequent expansion of arguments in 
Taylor series of their own, that the scheme given in (27.79) coincides with the 
Taylor expansion (27.68), i.e. 


7 go, 7, ko 
Yin = it hy; + Vi +i PEE 


up to terms in h?. 
To solve the ordinary differential equation 


du 
—= t 
Sa fu) 
for f = f(t), the explicit two-step finite difference scheme 
Unp = Uy + Bun + h(ufn aa vfn—1) 


may be used. Here, in the usual notation, h is the time step, t, = nh, u, = u(tn) 
and f, = f(Un,tn); % B, u, and v are constants. 


(a) A particular scheme has « = 1, B = 0, = 3/2 and v = —1/2. By considering 
Taylor expansions about t = t, for both u,,; and f,,;, show that this scheme 
gives errors of order h?. 

(b) Find the values of «, B, « and v that will give the greatest accuracy. 


Set up a finite difference scheme to solve the ordinary differential equation 


in the range 1 < x < 4, subject to the boundary conditions ¢(1) = 2 and 
do/dx = 2 at x = 4. Using N equal increments, Ax, in x, obtain the general 
difference equation and state how the boundary conditions are incorporated 
into the scheme. Setting Ax equal to the (crude) value 1, obtain the relevant 
simultaneous equations and so obtain rough estimates for #(2), (3) and (4). 

Finally, solve the original equation analytically and compare your numerical 
estimates with the accurate values. 
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27.21 


27.22 


27.23 


27.24 


Write a computer program that would solve, for a range of values of 4, the 
differential equation 

dy 1 

Y=, 0) =1, 

dx J x + Ay? 
using a third-order Runge-Kutta scheme. Consider the difficulties that might 
arise when 4 < 0. 
Use the isocline approach to sketch the family of curves that satisfies the non- 
linear first-order differential equation 


For some problems, numerical or algebraic experimentation may suggest the 
form of the complete solution. Consider the problem of numerically integrating 
the first-order wave equation 

ou + act = 0; 

ot Ox 
in which A is a positive constant. A finite difference scheme for this partial 
differential equation is 

u(p,n+1)—u(p,n) | ,u(p,n) — u(p — 1,n) 
+A 
At Ax 


where x = pAx and t = nAt, with p any integer and n a non-negative integer. 
The initial values are u(0,0) = 1 and u(p,0) = 0 for p # 0. 


=0, 


(a) Carry the difference equation forward in time for two or three steps and 
attempt to identify the pattern of solution. Establish the criterion for the 
method to be numerically stable. 

(b) Suggest a general form for u(p,n), expressing it in generator function form, 
ie. as “u(p,n) is the coefficient of s? in the expansion of G(n, s)’. 

(c) Using your form of solution (or that given in the answers!), obtain an 
explicit general expression for u(p,n) and verify it by direct substitution into 
the difference equation. 

(d) An analytic solution of the original PDE indicates that an initial distur- 
bance propagates undistorted. Under what circumstances would the differ- 
ence scheme reproduce that behaviour? 


In exercise 27.23 the difference scheme for solving 
ou Ou 
—+ —=0 
ot dx , 
in which A has been set equal to unity, was one-sided in both space (x) and 
time (t). A more accurate procedure (known as the Lax—Wendroff scheme) is 
HEPES) ¢ WOs LA Supt ) 
At 2Ax 
At | u(p + 1,1) — 2u(p,n) + u(p — 1,n) 
2 (Ax)? 
(a) Establish the orders of accuracy of the two finite difference approximations 
on the LHS of the equation. 
(b) Establish the accuracy with which the expression in the brackets approxi- 
mates 67u/0x?. 
(c) Show that the RHS of the equation is such as to make the whole difference 
scheme accurate to second order in both space and time. 
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27.25 


27.26 


Laplace’s equation, 
eV eV _ 0 
6x2 éy2”’ 


is to be solved for the region and boundary conditions shown in figure 27.7. 


V =80 


20 20 20 


N 
ll 
o 


Figure 27.7 Region, boundary values and initial guessed solution values. 


Starting from the given initial guess for the potential values V, and using the 
simplest possible form of relaxation, obtain a better approximation to the actual 
solution. Do not aim to be more accurate than + 0.5 units, and so terminate the 
process when subsequent changes would be no greater than this. 

Consider the solution, $(x,y), of Laplace’s equation in two dimensions using a 
relaxation method on a square grid with common spacing h. As in the main text, 
denote (xo + ih, yo + jh) by $j; Further, define $7" by 


omtn o 


Ox™ Oy” 


mn __ 
pi, Gics 


evaluated at (xo + ih, yo + jh). 
(a) Show that 
oi Ae 26:7 ats oy =.0), 
(b) Working up to terms of order h°, find Taylor series expansions, expressed in 
terms of the $/", for 
S40 = bitty + Pi. 
Sot = bij+t + Pij-1- 
(c) Find a corresponding expansion, to the same order of accuracy, for @i+1,j41+ 
¢i+1j-1 and hence show that 
Sie = dirt + Pitti + Pinter + Ginny 


has the form 
ht 
Agi) + Wis + Bin) + EUG) + Obi7 + Oi) 


(d) Evaluate the expression 4(S+i9 + So+)+S+4 and hence deduce that a possible 
relaxation scheme, good to the fifth order in h, is to recalculate each ¢;; as 
the weighted mean of the current values of its four nearest neighbours (each 
with weight ) and its four next-nearest neighbours (each with weight a): 
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27.10 HINTS AND ANSWERS 


27.27 


27.1 
27.3 
27.5 


27.7 


27.9 
27.11 


27.13 


The Schrodinger equation for a quantum mechanical particle of mass m moving 
in a one-dimensional harmonic oscillator potential V(x) = kx/2 is 
Py kx*p 


2m dx? 2 


For physically acceptable solutions, the wavefunction p(x) must be finite at x = 0, 
tend to zero as x > -+too and be normalised, so that f |p|? dx = 1. In practice, 
these constraints mean that only certain (quantised) values of E, the energy of 
the particle, are allowed. The allowed values fall into two groups: those for which 
y(0) = 0 and those for which p(0) # 0. 

Show that if the unit of length is taken as [fi2/(mk)]'/4 and the unit of energy 
is taken as h(k/m)'/?, then the Schrédinger equation takes the form 


= Ey. 


yp a 

3 +(2E'— y*)p =0 

Devise an outline computerised scheme, using Runge-Kutta integration, that will 
enable you to: 


(a) determine the three lowest allowed values of E; 
(b) tabulate the normalised wavefunction corresponding to the lowest allowed 
energy. 


You should consider explicitly: 


(i) the variables to use in the numerical integration; 

(ii) how starting values near y = 0 are to be chosen; 

(iii) how the condition on yp as y > +co is to be implemented; 

(iv) how the required values of E are to be extracted from the results of the 
integration; 

(v) how the normalisation is to be carried out. 


27.10 Hints and answers 


5.370. 

(a) € #0 and f’(&) #0 in general; (b) € = b, but f’(b) = 0 whilst f(b) 4 0. 
Interchange is formally needed for the first two steps, though in this case no error 
will result if it is not carried out; x1 12, xa = 2, 3 Ly xy 5. 

The quadratic equation is 27+ 4z+1=0;4+fB—3=x =a'4+ f' +3. 

With p = —2+4+ 3 and q = —2— \/3, B must be zero for i > 0 and a must be 
zero for i < 0; x; = 3(—2 + /3)' for i> 0, x; = 0 for i= 0, x; 3(-2 — /3)i 
for i <0. 

The error is 1% or less for |k| less than about 1.1. 

Exact values (6 s.f.) for p = 1,2,... ,6 are 1.570796, 1.178 097, 0.981 748, 0.859 029, 
0.773 126, 0.708 699. The Gauss—Chebyshev integration is in error by about 1% 
when n= p. 

Listed below are the relevant indefinite integrals F(y) of the distributions together 
with the functions € = &(): 


(a) y’, = vit; 
(b) 82, =p; 


(c) z{ {sin[xy/(2a)] + 1}, € = (2a/m) sin™'(2n — 1); 
(d) 5 exp y for y < 0, eae (—y)] for y > 0; € = In 2yn forO <n < ; 
€ =—In[2(1—n)] for $< <1. 
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27.15 


27.17 


27.19 


27.21 


27.23 


27.25 
27.27 


V =80 


—0 40.5} 41.8 46.7 48.4 46.7 41.8 40.5 « 


16.8 20.4 16.8 


3 
ll 
o 


Figure 27.8 The solution to exercise 27.25. 


1 — x?/2 + x4/8 — x°/48; 1.0000, 0.9950, 0.9802, 0.9560, 0.9231, 0.8825; exact 

solution y = exp(—x?/2). 

(b) ay = 23/12, ay = —4/3, a3 = 5/12. 

(c) by = 5/12, by = 2/3, b3 1/12. 

(d) (0.4) = 0.224 582, y(0.4) = 0.225 527 after correction. 

(a) The error is 5h3u9)/12 + O(h*). 

(b) a 4,p8 =5,u=4andv=2 

For 4 positive the solutions are (boringly) monotonic functions of x. With y(0) 

given, there are no real solutions at all for any negative /! 

(a) Setting AAt = cAx gives, for example, u(0,2) = (1 — c)’, u(1,2) = 2c(1 — ¢), 
u(2,2) = c?. For stability, 0 <¢ <1. 

(b) G(n,s) = [(1—c)+cs]" forO<p<n. 

(c) [nll —c)"Pe"]/[p'(n — p) 1]. 

(d) When c = 1 and the difference equation becomes u(p,n + 1) = u(p — 1,n). 

See figure 27.8. 

If x = ay then 


Solutions will be either symmetric or antisymmetric with p(0) £0 but p’(0) =0 
for the former and vice versa for the latter. Integration to a largish but finite 
value of y followed by an interpolation procedure to estimate the values of E 
that lead to y(oo) = 0 needs to be incorporated. Simple numerical integration 
such as Simpson’s rule will suffice for the normalisation integral. The solutions 
should be 4 = 1,3,5,... . 
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Group theory 


For systems that have some degree of symmetry, full exploitation of that symmetry 
is desirable. Significant physical results can sometimes be deduced simply by a 
study of the symmetry properties of the system under investigation. Consequently 
it becomes important, for such a system, to identify all those operations (rotations, 
reflections, inversions) that carry the system into a physically indistinguishable 
copy of itself. 

The study of the properties of the complete set of such operations forms 
one application of group theory. Though this is the aspect of most interest to 
the physical scientist, group theory itself is a much larger subject and of great 
importance in its own right. Consequently we leave until the next chapter any 
direct applications of group theoretical results and concentrate on building up 
the general mathematical properties of groups. 


28.1 Groups 


As an example of symmetry properties, let us consider the sets of operations, 
such as rotations, reflections, and inversions, that transform physical objects, for 
example molecules, into physically indistinguishable copies of themselves, so that 
only the labelling of identical components of the system (the atoms) changes in 
the process. For differently shaped molecules there are different sets of operations, 
but in each case it is a well-defined set, and with a little practice all members of 
each set can be identified. 

As simple examples, consider (a) the hydrogen molecule, and (b) the ammonia 
molecule illustrated in figure 28.1. The hydrogen molecule consists of two atoms 
H of hydrogen and is carried into itself by any of the following operations: 


(i) any rotation about its long axis; 
(ii) rotation through 2 about an axis perpendicular to the long axis and 
passing through the point M that lies midway between the atoms; 
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N 
H 
H , H 
(a) (b) H 


Figure 28.1 (a) The hydrogen molecule, and (b) the ammonia molecule. 


(iii) inversion through the point M; 
(iv) reflection in the plane that passes through M and has its normal parallel 
to the long axis. 


These operations collectively form the set of symmetry operations for the hydro- 
gen molecule. 

The somewhat more complex ammonia molecule consists of a tetrahedron with 
an equilateral triangular base at the three corners of which lie hydrogen atoms 
H, whilst a nitrogen atom N is sited at the fourth vertex of the tetrahedron. The 
set of symmetry operations on this molecule is limited to rotations of 7/3 and 
2n/3 about the axis joining the centroid of the equilateral triangle to the nitrogen 
atom, and reflections in the three planes containing that axis and each of the 
hydrogen atoms in turn. However, if the nitrogen atom could be replaced by a 
fourth hydrogen atom, and all interatomic distances equalised in the process, the 
number of symmetry operations would be greatly increased. 

Once all the possible operations in any particular set have been identified, it 
must follow that the result of applying two such operations in succession will be 
identical to that obtained by the sole application of some third (usually different) 
operation in the set — for if it were not, a new member of the set would have 
been found, contradicting the assumption that all members have been identified. 

Such observations introduce two of the main considerations relevant to decid- 
ing whether a set of objects, here the rotation, reflection and inversion operations, 
qualifies as a group in the mathematically tightly defined sense. These two consid- 
erations are (i) whether there is some law for combining two members of the set, 
and (ii) whether the result of the combination is also a member of the set. The 
obvious rule of combination has to be that the second operation is carried out 
on the system that results from application of the first operation, and we have 
already seen that the second requirement is satisfied by the inclusion of all such 
operations in the set. However, for a set to qualify as a group, more than these 
two conditions have to be satisfied, as will now be made clear. 
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28.1.1 Definition of a group 


A group G is a set of elements {X,Y,...}, together with a rule for combining 
them that associates with each ordered pair X, Y a ‘product’ or combination law 
X e Y for which the following conditions must be satisfied. 


(i) For every pair of elements X, Y that belongs to G, the product X e Y also 
belongs to G. (This is known as the closure property of the group.) 
(ii) For all triples X, Y, Z the associative law holds; in symbols, 


Xe(YeZ)=(XeY)eZ. (28.1) 
(iii) There exists a unique element I, belonging to G, with the property that 
leX=X=Xel (28.2) 


for all X belonging to G. This element J is known as the identity element 
of the group. 

(iv) For every element X of G, there exists an element X—!, also belonging to 
G, such that 


XteX¥ =1=XeX!. (28.3) 
X7! is called the inverse of X. 


An alternative notation in common use is to write the elements of a group G 
as the set {G;,G,...} or, more briefly, as {G;}, a typical element being denoted 
by Gj. 

It should be noticed that, as given, the nature of the operation e is not stated. It 
should also be noticed that the more general term element, rather than operation, 
has been used in this definition. We will see that the general definition of a 
group allows as elements not only sets of operations on an object but also sets of 
numbers, of functions and of other objects, provided that the interpretation of e 
is appropriately defined. 

In one of the simplest examples of a group, namely the group of all integers 
under addition, the operation e is taken to be ordinary addition. In this group the 
role of the identity I is played by the integer 0, and the inverse of an integer X is 
—X. That requirements (i) and (i1) are satisfied by the integers under addition is 
trivially obvious. A second simple group, under ordinary multiplication, is formed 
by the two numbers | and —1; in this group, closure is obvious, 1 is the identity 
element, and each element is its own inverse. 

It will be apparent from these two examples that the number of elements in a 
group can be either finite or infinite. In the former case the group is called a finite 
group and the number of elements it contains is called the order of the group, 
which we will denote by g; an alternative notation is |G| but has obvious dangers 
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if matrices are involved. In the notation in which G = {G,, G2,...,G,} the order 
of the group is clearly n. 

As we have noted, for the integers under addition zero is the identity. For 
the group of rotations and reflections, the operation of doing nothing, ie. the 
null operation, plays this role. This latter identification may seem artificial, but 
it is an operation, albeit trivial, which does leave the system in a physically 
indistinguishable state, and needs to be included. One might add that without it 
the set of operations would not form a group and none of the powerful results 
we will derive later in this and the next chapter could be justifiably applied to 
give deductions of physical significance. 

In the examples of rotations and reflections mentioned earlier, e has been taken 
to mean that the left-hand operation is carried out on the system that results 
from application of the right-hand operation. Thus 


Z=XeyY (28.4) 


means that the effect on the system of carrying out Z is the same as would 
be obtained by first carrying out Y and then carrying out X. The order of the 
operations should be noted; it is arbitrary in the first instance but, once chosen, 
must be adhered to. The choice we have made is dictated by the fact that most 
of our applications involve the effect of rotations and reflections on functions of 
space coordinates, and it is usual, and our practice in the rest of this book, to 
write operators acting on functions to the left of the functions. 

It will be apparent that for the above-mentioned group, integers under ordinary 
addition, it is true that 


YeX=XeY (28.5) 


for all pairs of integers X, Y. If any two particular elements of a group satisfy 
(28.5), they are said to commute under the operation e; if all pairs of elements in 
a group satisfy (28.5), then the group is said to be Abelian. The set of all integers 
forms an infinite Abelian group under (ordinary) addition. 

As we show below, requirements (iii) and (iv) of the definition of a group 
are over-demanding (but self-consistent), since in each of equations (28.2) and 
(28.3) the second equality can be deduced from the first by using the associativity 
required by (28.1). The mathematical steps in the following arguments are all 
very simple, but care has to be taken to make sure that nothing that has not 
yet been proved is used to justify a step. For this reason, and to act as a model 
in logical deduction, a reference in Roman numerals to the previous result, 
or to the group definition used, is given over each equality sign. Such explicit 
detailed referencing soon becomes tiresome, but it should always be available if 
needed. 
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> Using only the first equalities in (28.2) and (28.3), deduce the second ones. 


Consider the expression X~! e (X e X—'); 
xte(xex) ® te xyex? 
(ii) oa 


WW). xt 


But X7! belongs to G, and so from (iv) there is an element U in G such that 


Uex =I. (v) 
Form the product of U with the first and last expressions in (28.6) to give 


Ue (X10 (Xe X-)) =e x Y 1, 


Transforming the left-hand side of this equation gives 


Ue(X 'e(XeX~')) ) (UeX')e(XeXx') 
My) ITe(XeX"') 
ee a 
Comparing (28.7), (28.8) shows that 
XexX'=I], (ivy 


i.e. the second equality in group definition (iv). Similarly 


Xel OW) y ocx! eX) (i) (XeX ex 
(ix) | eX 
(iti) (iii’) 


i.e. the second equality in group definition (iii). < 


(28.6) 


(28.7) 


(28.8) 


The uniqueness of the identity element J can also be demonstrated rather than 


assumed. Suppose that I’, belonging to G, also has the property 


DexX=X=Xel' for all X belonging to G. 
Take X as I, then 
Tel=I. 
Further, from (iii’), 


X=Xel for all X belonging to G, 
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and setting X =I’ gives 


=l'el. (28.10) 


It then follows from (28.9), (28.10) that J = I’, showing that in any particular 
group the identity element is unique. 

In a similar way it can be shown that the inverse of any particular element 
is unique. If U and V are two postulated inverses of an element X of G, by 
considering the product 


Ue(XeV)=(UeX)eY, 


it can be shown that U = V. The proof is left to the reader. 
Given the uniqueness of the inverse of any particular group element, it follows 
that 


(UeVe-.-eYeZ)o(Z ey 'e...oV'eU') 
=(UeVe..-eY)e(ZeZ')e(Y 'e---eV tet!) 
=(UeVe..-eY)e(Y 'e---eV!et') 


=I, 


where use has been made of the associativity and of the two equations Z e Z~! =] 
and I e X = X. Thus the inverse of a product is the product of the inverses in 
reverse order, i.e. 


(UeVe.--e¥ eZ)! =(Z ley 'e.--eV'et"}). (28.11) 


Further elementary results that can be obtained by arguments similar to those 
above are as follows. 


(i) Given any pair of elements X, Y belonging to G, there exist unique 
elements U, V, also belonging to G, such that 


XeU=Y and VeX=Y. 


Clearly U = X-'e Y, and V = Y e X~!, and they can be shown to be 
unique. This result is sometimes called the division axiom. 
(ii) The cancellation law can be stated as follows. If 


XeY=XeZ 
for some X belonging to G, then Y = Z. Similarly, 
YeX=ZexX 


implies the same conclusion. 
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Figure 28.2 Reflections in the three perpendicular bisectors of the sides of 
an equilateral triangle take the triangle into itself. 


(iii) Forming the product of each element of G with a fixed element X of G 
simply permutes the elements of G; this is often written symbolically as 
GexX =G. If this were not so, and X e Y and X eZ were not different 
even though Y and Z were, application of the cancellation law would lead 
to a contradiction. This result is called the permutation law. 


In any finite group of order g, any element X when combined with itself to 
form successively X? = X eX, X3 = Xe X?,... will, after at most g—1 such 
combinations, produce the group identity I. Of course X?, X%, ... are some of 
the original elements of the group, and not new ones. If the actual number of 
combinations needed is m—1, i.e. X” =I, then m is called the order of the element 
X in G. The order of the identity of a group is always 1, and that of any other 
element of a group that is its own inverse is always 2. 


> Determine the order of the group of (two-dimensional ) rotations and reflections that take 
a plane equilateral triangle into itself and the order of each of the elements. The group is 


usually known as 3m (to physicists and crystallographers ) or C3, (to chemists ). 


There are two (clockwise) rotations, by 2x/3 and 42/3, about an axis perpendicular to 
the plane of the triangle. In addition, reflections in the perpendicular bisectors of the three 
sides (see figure 28.2) have the defining property. To these must be added the identity 
operation. Thus in total there are six distinct operations and so g = 6 for this group. 
To reproduce the identity operation either of the rotations has to be applied three times, 
whilst any of the reflections has to be applied just twice in order to recover the original 
situation. Thus each rotation element of the group has order 3, and each reflection element 
has order 2. < 


A so-called cyclic group is one for which all members of the group can be 
generated from just one element X (say). Thus a cyclic group of order g can be 
written as 


PONT OT 
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It is clear that cyclic groups are always Abelian and that each element, apart 
from the identity, has order g, the order of the group itself. 


28.1.2 Further examples of groups 


In this section we consider some sets of objects, each set together with a law of 
combination, and investigate whether they qualify as groups and, if not, why not. 

We have already seen that the integers form a group under ordinary addition, 
but it is immediately apparent that (even if zero is excluded) they do not do 
so under ordinary multiplication. Unity must be the identity of the set, but the 
requisite inverse of any integer n, namely 1/n, does not belong to the set of 
integers for any n other than unity. 

Other infinite sets of quantities that do form groups are the sets of all real 
numbers, or of all complex numbers, under addition, and of the same two sets 
excluding 0 under multiplication. All these groups are Abelian. 

Although subtraction and division are normally considered the obvious coun- 
terparts of the operations of (ordinary) addition and multiplication, they are not 
acceptable operations for use within groups since the associative law, (28.1), does 
not hold. Explicitly, 


X-(Y¥ —Z)4(X-Y)-Z, 
X+(Y +Z)4A(X+Y)+Z. 


From within the field of all non-zero complex numbers we can select just those 
that have unit modulus, i.e. are of the form e” where 0 < 0 < 2z, to form a 
group under multiplication, as can easily be verified: 


eit x ei2 = il +02) (closure), 
eid = (identity), 
el2n-0) y¢ gi — idm = pi 1 (inverse). 


Closely related to the above group is the set of 2 x 2 rotation matrices that take 
the form 
cos? —siné 
M(0) = . 
(8) ( sin@ —_cos@ ) 


where, as before, 0 < 0 < 2x. These form a group when the law of combination 
is that of matrix multiplication. The reader can easily verify that 


M(O)M(@) = M(@+ ¢) (closure), 
M(0) = lb (identity), 
M(2x —0) = M~!(@) _ (inverse). 


Here |, is the unit 2 x 2 matrix. 
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28.2 Finite groups 


Whilst many properties of physical systems (e.g. angular momentum) are related 
to the properties of infinite, and, in particular, continuous groups, the symmetry 
properties of crystals and molecules are more intimately connected with those of 
finite groups. We therefore concentrate in this section on finite sets of objects that 
can be combined in a way satisfying the group postulates. 

Although it is clear that the set of all integers does not form a group under 
ordinary multiplication, restricted sets can do so if the operation involved is multi- 
plication (mod N) for suitable values of N; this operation will be explained below. 

As a simple example of a group with only four members, consider the set S 
defined as follows: 


S = {1,3,5,7} under multiplication (mod 8). 


To find the product (mod 8) of any two elements, we multiply them together in 
the ordinary way, and then divide the answer by 8, treating the remainder after 
doing so as the product of the two elements. For example, 5 x 7 = 35, which on 
dividing by 8 gives a remainder of 3. Clearly, since Y x Z = Z x Y, the full set 
of different products is 


Pet. AK S39 Te 55. TRIS 
BYR 3xha7 Sep, 

5x5=1, 5x7=3, 

TRTSA: 


The first thing to notice is that each multiplication produces a member of the 
original set, i.e. the set is closed. Obviously the element 1 takes the role of the 
identity, ic. 1 x Y = Y for all members Y of the set. Further, for each element Y 
of the set there is an element Z (equal to Y, as it happens, in this case) such that 
Y x Z =1,1e. each element has an inverse. These observations, together with the 
associativity of multiplication (mod 8), show that the set S is an Abelian group 
of order 4. 

It is convenient to present the results of combining any two elements of a 
group in the form of multiplication tables — akin to those which used to appear in 
elementary arithmetic books before electronic calculators were invented! Written 
in this much more compact form the above example is expressed by table 28.1. 
Although the order of the two elements being combined does not matter here 
because the group is Abelian, we adopt the convention that if the product in a 
general multiplication table is written X e Y then X is taken from the left-hand 
column and Y is taken from the top row. Thus the bold ‘7’ in the table is the 
result of 3 x 5, rather than of 5 x 3. 

Whilst it would make no difference to the basic information content in a table 
to present the rows and columns with their headings in random orders, it is 
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YON wW Re 


Table 28.1 The table of products for the elements of the group S = {1,3,5,7} 
under multiplication (mod 8). 


usual to list the elements in the same order in both the vertical and horizontal 
headings in any one table. The actual order of the elements in the common list, 
whilst arbitrary, is normally chosen to make the table have as much symmetry as 
possible. This is initially a matter of convenience, but, as we shall see later, some 
of the more subtle properties of groups are revealed by putting next to each other 
elements of the group that are alike in certain ways. 

Some simple general properties of group multiplication tables can be deduced 
immediately from the fact that each row or column constitutes the elements of 
the group. 


(i) Each element appears once and only once in each row or column of the 
table; this must be so since G e X =G (the permutation law) holds. 

(ii) The inverse of any element Y can be found by looking along the row 
in which Y appears in the left-hand column (the Yth row), and noting 
the element Z at the head of the column (the Zth column) in which 
the identity appears as the table entry. An immediate corollary is that 
whenever the identity appears on the leading diagonal, it indicates that 
the corresponding header element is of order 2 (unless it happens to be 
the identity itself). 

(iii) For any Abelian group the multiplication table is symmetric about the 
leading diagonal. 


To get used to the ideas involved in using group multiplication tables, we now 
consider two more sets of integers under multiplication (mod N): 


S’ = {1,5,7,11} under multiplication (mod 24), and 
S" = {1,2,3,4} under multiplication (mod 5). 


These have group multiplication tables 28.2(a) and (b) respectively, as the reader 
should verify. 

If tables 28.1 and 28.2(a) for the groups S and S’ are compared, it will be seen 
that they have essentially the same structure, ie if the elements are written as 
{1,A, B,C} in both cases, then the two tables are each equivalent to table 28.3. 

For S,] = 1, A = 3, B =5, C =7 and the law of combination is multiplication 
(mod 8), whilst for S’, J = 1, A= 5, B= 7, C = 11 and the law of combination 
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1 5 7 it 
1 5 7 it 
(a) 5 1 11 #7 (b) 
7, “Td ch 5 
ll 67 5 1 


Table 28.2 (a) The multiplication table for the group S’ = {1,5,7, 11} under 
multiplication (mod 24). (b) The multiplication table for the group S” = 
{1, 2,3,4} under multiplication (mod 5). 


Table 28.4 The group table for the set {1,i,—1,—i} under ordinary multipli- 
cation of complex numbers. 


is multiplication (mod 24). However, the really important point is that the two 
groups S and S’ have equivalent group multiplication tables — they are said to 
be isomorphic, a matter to which we will return more formally in section 28.5. 


> Determine the behaviour of the set of four elements 
{1, i, —1, —i} 
under the ordinary multiplication of complex numbers. Show that they form a group and 


determine whether the group is isomorphic to either of the groups S (itself isomorphic to 
S’) and S" defined above. 


That the elements form a group under the associative operation of complex multiplication 
is immediate; there is an identity (1), each possible product generates a member of the set 
and each element has an inverse (1, —i, —1, i, respectively). The group table has the form 
shown in table 28.4. 

We now ask whether this table can be made to look like table 28.3, which is the 
standardised form of the tables for S and S’. Since the identity element of the group 
(1) will have to be represented by J, and ‘1’ only appears on the leading diagonal twice 
whereas I appears on the leading diagonal four times in table 28.3, it is clear that no 
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i -1 -i 

i -1 -i 
-1 -i 1 
-i 1 i 

1 i —1 


Table 28.5 A comparison between tables 28.4 and 28.2(b), the latter with its 
columns reordered. 


mm Q Bw 
Bar qa 


Table 28.6 The common structure exemplified by tables 28.4 and 28.2(b), the 
latter with its columns reordered. 


amount of relabelling (or, equivalently, no allocation of the symbols A, B, C, amongst 
i, —1, —i) can bring table 28.4 into the form of table 28.3. We conclude that the group 
{1,i,-1,—i} is not isomorphic to S or S’. An alternative way of stating the observation is 
to say that the group contains only one element of order 2 whilst a group corresponding 
to table 28.3 contains three such elements. 

However, if the rows and columns of table 28.2(b) — in which the identity does appear 
twice on the diagonal and which therefore has the potential to be equivalent to table 28.4 — 
are rearranged by making the heading order 1, 2, 4, 3 then the two tables can be compared 
in the forms shown in table 28.5. They can thus be seen to have the same structure, namely 
that shown in table 28.6. 

We therefore conclude that the group of four elements {1, i, —1,—i} under ordinary mul- 
tiplication of complex numbers is isomorphic to the group {1,2,3,4} under multiplication 
(mod 5). < 


What we have done does not prove it, but the two tables 28.3 and 28.6 are in 
fact the only possible tables for a group of order 4, i.e. a group containing exactly 
four elements. 


28.3, Non-Abelian groups 


So far, all the groups for which we have constructed multiplication tables have 
been based on some form of arithmetic multiplication, a commutative operation, 
with the result that the groups have been Abelian and the tables symmetric 
about the leading diagonal. We now turn to examples of groups in which some 
non-commutation occurs. It should be noted, in passing, that non-commutation 
cannot occur throughout a group, as the identity always commutes with any 
element in its group. 
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As a first example we consider again as elements of a group the two-dimensional 
operations which transform an equilateral triangle into itself (see the end of 
subsection 28.1.1). It has already been shown that there are six such operations: 
the null operation, two rotations (by 27/3 and 42/3 about an axis perpendicular 
to the plane of the triangle) and three reflections in the perpendicular bisectors 
of the three sides. To abbreviate we will denote these operations by symbols as 
follows. 


(i) I is the null operation. 
(ii) R and R’ are (clockwise) rotations by 27/3 and 47/3 respectively. 
(iii) K, L, M are reflections in the three lines indicated in figure 28.2. 


Some products of the operations of the form X e Y (where it will be recalled 
that the symbol e means that the second operation X is carried out on the system 
resulting from the application of the first operation Y) are easily calculated: 


ReR=R, ReR=R, ReR=I=ReR 


(28.12) 
KeK=LeL=MeM=l. 


Others, such as K e M, are more difficult, but can be found by a little thought, 
or by making a model triangle or drawing a sequence of diagrams such as those 
following. 


@ x 
KeM = = 
Oo x @ Oo 
showing that K e M = R’. In the same way, 
@ @ O @ 
MeK = M = = R 
O x x O x t ) ie) x 
shows that MeK = R, and 
( O @ t 
ReL = R = = K 
e) 4 O x ‘s 2) O x 


shows that Re L=K. 

Proceeding in this way we can build up the complete multiplication table 
(table 28.7). In fact, it is not necessary to draw any more diagrams, as all 
remaining products can be deduced algebraically from the three found above and 
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I R RK kK L M 
I R R kK L M 
R R I M kK L 
RoI R L M K 
K L M I R R 
L M kK R I R 
M K L R R I 


Table 28.7. The group table for the two-dimensional symmetry operations on 
an equilateral triangle. 


the more self-evident results given in (28.12). A number of things may be noticed 
about this table. 


(i) It is not symmetric about the leading diagonal, indicating that some pairs 
of elements in the group do not commute. 

(ii) There is some symmetry within the 3 x 3 blocks that form the four quarters 
of the table. This occurs because we have elected to put similar operations 
close to each other when choosing the order of table headings — the two 
rotations (or three if I is viewed as a rotation by 0/3) are next to each 
other, and the three reflections also occupy adjacent columns and rows. 
We will return to this later. 


That two groups of the same order may be isomorphic carries over to non- 
Abelian groups. The next two examples are each concerned with sets of six 
objects; they will be shown to form groups that, although very different in nature 
from the rotation-reflection group just considered, are isomorphic to it. 

We consider first the set M of six orthogonal 2 x 2 matrices given by 


=a B 1 = 
I= 1 0 A= 2 2 B= De 22) 
0 1 SPB h es ww _t 
5) 2 2 
1 V3 1 3 
-1 0 3 = 7 
C= D= z ={ 2% 
0 1 oe es | vB 1 
2 2 2 2 


the combination law being that of ordinary matrix multiplication. Here we use 
italic, rather than the sans serif used for matrices elsewhere, to emphasise that 
the matrices are group elements. 

Although it is tedious to do so, it can be checked that the product of any 
two of these matrices, in either order, is also in the set. However, the result is 
generally different in the two cases, as matrix multiplication is non-commutative. 
The matrix I clearly acts as the identity element of the set, and during the checking 
for closure it is found that the inverse of each matrix is contained in the set, J, 
C, D and E being their own inverses. The group table is shown in table 28.8. 
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moavann 
Amor Bae 
Samar sy 
aor omaga 
Beaman» 
s~PewQoRm 


moavan 


Table 28.8 The group table, under matrix multiplication, for the set M of 
six orthogonal 2 x 2 matrices given by (28.13). 


The similarity to table 28.7 is striking. If {R,R’,K,L,M} of that table are 
replaced by {A, B,C, D, E} respectively, the two tables are identical, without even 
the need to reshuffle the rows and columns. The two groups, one of reflections 
and rotations of an equilateral triangle, the other of matrices, are isomorphic. 

Our second example of a group isomorphic to the same rotation—reflection 
group is provided by a set of functions of an undetermined variable x. The 
functions are as follows: 


fils) =x, f2(x) = 1/1 — x), f3(x) = (x — 1)/x, 
fax) =1/x,  fs(x) = 1—-x, fo(x) = x/(x — 1), 


and the law of combination is 
Fil) © Fix) = FilFj), 

i.e. the function on the right acts as the argument of the function on the left to 
produce a new function of x. It should be emphasised that it is the functions 
that are the elements of the group. The variable x is the ‘system’ on which they 
act, and plays much the same role as the triangle does in our first example of a 
non-Abelian group. 

To show an explicit example, we calculate the product f¢ e f;. The product 


will be the function of x obtained by evaluating y/(y — 1), when y is set equal to 
(x — 1)/x. Explicitly 


fells) = REDE = 1 x= fats 
Thus f¢ ¢ f3 = fs. Further examples are 
Se ane 1 x—1 : 
1202 Geary oe 
and 
—1 
ities aan a 7 =x=hi (28.14) 
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The multiplication table for this set of six functions has all the necessary proper- 
ties to show that they form a group. Further, if the symbols f1, f2, f3, fa, fs, fo are 
replaced by J,A,B,C,D,E respectively the table becomes identical to table 28.8. 
This justifies our earlier claim that this group of functions, with argument sub- 
stitution as the law of combination, is isomorphic to the group of reflections and 
rotations of an equilateral triangle. 


28.4 Permutation groups 


The operation of rearranging n distinct objects amongst themselves is called a 
permutation of degree n, and since many symmetry operations on physical systems 
can be viewed in that light, the properties of permutations are of interest. For 
example, the symmetry operations on an equilateral triangle, to which we have 
already given much attention, can be considered as the six possible rearrangements 
of the marked corners of the triangle amongst three fixed points in space, much 
as in the diagrams used to compute table 28.7. In the same way, the symmetry 
operations on a cube can be viewed as a rearrangement of its corners amongst 
eight points in space, albeit with many constraints, or, with fewer complications, 
as a rearrangement of its body diagonals in space. The details will be left until 
we review the possible finite groups more systematically. 

The notations and conventions used in the literature to describe permutations 
are very varied and can easily lead to confusion. We will try to avoid this by using 
letters a,b,c,... (rather than numbers) for the objects that are rearranged by a 
permutation and by adopting, before long, a ‘cycle notation’ for the permutations 
themselves. It is worth emphasising that it is the permutations, 1.e. the acts of 
rearranging, and not the objects themselves (represented by letters) that form 
the elements of permutation groups. The complete group of all permutations of 
degree n is usually denoted by S, or Z,. The number of possible permutations of 
degree n is n!, and so this is the order of S),. 

Suppose the ordered set of six distinct objects {a b cd e f} is rearranged by 
some process into {b e f adc}; then we can represent this mathematically as 


Ofabcdef }={befadc}, 
where @ is a permutation of degree 6. The permutation 0 can be denoted by 
(2 5 6 1 4 3], since the first object, a, is replaced by the second, b, the second 


object, b, is replaced by the fifth, e, the third by the sixth, f, etc. The equation 
can then be written more explicitly as 


Ofabcde f}=[25614 3]{fabcdef}={befadct. 


If ¢ is a second permutation, also of degree 6, then the obvious interpretation of 
the product ¢ e 0 of the two permutations is 


oeOfabcdef}=¢d(Ofabcde f}). 


1056 


28.4 PERMUTATION GROUPS 


Suppose that ¢ is the permutation [4 5 3 6 2 1]; then 


bedlabcde ft =(45362 1125614 3fabede f} 
=[453621{befadc} 
={adfceb} 
=[146352]{abedeft. 


Written in terms of the permutation notation this result is 
[45362 1125614 3)=[1 4635 2]. 


A concept that is very useful for working with permutations is that of decom- 
position into cycles. The cycle notation is most easily explained by example. For 
the permutation 0 given above: 


the Ist object, a, has been replaced by the 2nd, b; 
the 2nd object, b, has been replaced by the Sth, e; 
the Sth object, e, has been replaced by the 4th, d; 
the 4th object, d, has been replaced by the Ist, a. 


This brings us back to the beginning of a closed cycle, which is conveniently 
represented by the notation (1 2 5 4), in which the successive replacement 
positions are enclosed, in sequence, in parentheses. Thus (1 2 5 4) means 2nd 
— Ist, 5th > 2nd, 4th — Sth, lst — 4th. It should be noted that the object 
initially in the first listed position replaces that in the final position indicated in 
the bracket — here ‘a’ is put into the fourth position by the permutation. Clearly 
the cycle (5 4 1 2), or any other that involved the same numbers in the same 
relative order, would have exactly the same meaning and effect. The remaining 
two objects, c and f, are interchanged by @ or, more formally, are rearranged 
according to a cycle of length 2, a transposition, represented by (3 6). Thus the 
complete representation (specification) of 0 is 


6=(1254)(3 6). 


The positions of objects that are unaltered by a permutation are either placed by 
themselves in a pair of parentheses or omitted altogether. The former is recom- 
mended as it helps to indicate how many objects are involved — important when 
the object in the last position is unchanged, or the permutation is the identity, 
which leaves all objects unaltered in position! Thus the identity permutation of 
degree 6 is 


T = (1)(2)(3)(4)(5)(6), 


though in practice it is often shortened to (1). 
It will be clear that the cycle representation is unique, to within the internal 
absolute ordering of the numbers in each bracket as already noted, and that 
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each number appears once and only once in the representation of any particular 
permutation. 

The order of any permutation of degree n within the group S, can be read off 
from the cyclic representation and is given by the lowest common multiple (LCM) 
of the lengths of the cycles. Thus J has order 1, as it must, and the permutation 
@ discussed above has order 4 (the LCM of 4 and 2). 

Expressed in cycle notation our second permutation ¢ is (3)(1 4 6)(2 5), and 
the product ¢ e 0 is calculated as 


(3)(1 4 6)(2 5) 0 (125 4)(3 6){abcde f} =(3)(1 4 6)(2 5){be f adc} 
={adfceb} 
= (1)(5)(2 43 6){abcde f}. 
i.e. expressed as a relationship amongst the elements of the group of permutations 


of degree 6 (not yet proved as a group, but reasonably anticipated), this result 
reads 


(3)(1 4 6)(2 5) e(1 2 5 4)(3 6) = (1)(5)(2 4 3 6). 


We note, for practice, that ¢ has order 6 (the LCM of 1, 3, and 2) and that the 
product ¢e @ has order 4. 

The number of elements in the group S, of all permutations of degree n is 
n! and clearly increases very rapidly as n increases. Fortunately, to illustrate the 
essential features of permutation groups it is sufficient to consider the case n = 3, 
which involves only six elements. They are as follows (with labelling which the 
reader will by now recognise as anticipatory): 


I =(1)(2)3) A=(123) B=(1 32) 
C= (123) D=(3)(12) E=(2)(1 3) 
It will be noted that A and B have order 3, whilst C, D and E have order 2. As 


perhaps anticipated, their combination products are exactly those corresponding 
to table 28.8, I, C, D and E being their own inverses. For example, putting in all 
steps explicitly, 
DeC{a bc} = (3)(1 2) e (1)(2 3){a b c} 

= (3)(12){a c b} 

= {cab} 

= (3 2 1){a bc} 

= (1 3 2){a bc} 

= Blab c}. 
In brief, the six permutations belonging to S; form yet another non-Abelian group 
isomorphic to the rotation—-reflection symmetry group of an equilateral triangle. 
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28.5 Mappings between groups 


Now that we have available a range of groups that can be used as examples, 
we return to the study of more general group properties. From here on, when 
there is no ambiguity we will write the product of two elements, X e Y, simply 
as XY, omitting the explicit combination symbol. We will also continue to use 
‘multiplication’ as a loose generic name for the combination process between 
elements of a group. 

If G and G’ are two groups, we can study the effect of a mapping 


O:Gog 


of G onto G’. If X is an element of G we denote its image in G’ under the mapping 
® by X' = O(X). 

A technical term that we have already used is isomorphic. We will now define 
it formally. Two groups G = {X,Y,...} and G’ = {X’,Y’,...} are said to be 
isomorphic if there is a one-to-one correspondence 


XoxX, YoY',--- 
between their elements such that 
XY=Z implies X'Y’=Z' 


and vice versa. 

In other words, isomorphic groups have the same (multiplication) structure, 
although they may differ in the nature of their elements, combination law and 
notation. Clearly if groups G and G’ are isomorphic, and G and G” are isomorphic, 
then it follows that G’ and G” are isomorphic. We have already seen an example of 
four groups (of functions of x, of orthogonal matrices, of permutations and of the 
symmetries of an equilateral triangle) that are isomorphic, all having table 28.8 
as their multiplication table. 

Although our main interest is in isomorphic relationships between groups, the 
wider question of mappings of one set of elements onto another is of some 
importance, and we start with the more general notion of a homomorphism. 


Let G and G’ be two groups and ® a mapping of G > G’. If for every pair of 
elements X and Y inG 


(XY) =X’'Y’ 
then ® is called a homomorphism, and G' is said to be a homomorphic image of G. 


The essential defining relationship, expressed by (XY) = X'Y’, is that the 
same result is obtained whether the product of two elements is formed first and 
the image then taken or the images are taken first and the product then formed. 
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Three immediate consequences of the above definition are proved as follows. 
(i) If I is the identity of G then 1X = X for all X in G. Consequently 


X' = (IXY =1'X, 


for all X’ in G’. Thus I’ is the identity in G’. In words, the identity element 
of G maps into the identity element of G’. 
(ii) Further, 
P= (XX71y’ = X(XTY. 


That is, (X—!)’ = (X’)!. In words, the image of an inverse is the same 
element in G’ as the inverse of the image. 
(iii) If element X in G is of order m, ie. I = X", then 


T= (Xs = (XX TY = XK a. SH XX. X! 
=—~_Y-——"" 
m factors 
In words, the image of an element has the same order as the element. 


What distinguishes an isomorphism from the more general homomorphism are 
the requirements that in an isomorphism: 


(1) different elements in G must map into different elements in G’ (whereas in 
a homomorphism several elements in G may have the same image in G’), 
that is, x’ = y’ must imply x = y; 

(II) any element in G’ must be the image of some element in G. 


An immediate consequence of (I) and result (iii) for homomorphisms is that 
isomorphic groups each have the same number of elements of any given order. 

For a general homomorphism, the set of elements of G whose image in G’ 
is I’ is called the kernel of the homomorphism; this is discussed further in the 
next section. In an isomorphism the kernel consists of the identity J alone. To 
illustrate both this point and the general notion of a homomorphism, consider 
a mapping between the additive group of real numbers ® and the multiplicative 
group of complex numbers with unit modulus, U(1). Suppose that the mapping 
R — U(1) is 

®:x > e; 


then this is a homomorphism since 


(x y)! ellxty) = eXelv — x'y’. 


However, it is not an isomorphism because many (an infinite number) of the 
elements of ® have the same image in U(1). For example, z,3z,5z,... in ® all 
have the image —1 in U(1) and, furthermore, all elements of R of the form 2zn, 
where n is an integer, map onto the identity element in U(1). The latter set forms 
the kernel of the homomorphism. 
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(b) 


moan er 
QAmUa wee 
Samra Bw 
aoe os maa 
Bera may»yd 
SRBBQORH 


Table 28.9 Reproduction of (a) table 28.8 and (b) table 28.3 with the relevant 
subgroups shown in bold. 


For the sake of completeness, we add that a homomorphism for which (I) above 
holds is said to be a monomorphism (or an isomorphism into), whilst a homomor- 
phism for which (II) holds is called an epimorphism (or an isomorphism onto). If, 
in either case, the other requirement is met as well then the monomorphism or 
epimorphism is also an isomorphism. 

Finally, if the initial and final groups are the same, G = G’, then the isomorphism 
G —> 4G’ is termed an automorphism. 


28.6 Subgroups 


More detailed inspection of tables 28.8 and 28.3 shows that not only do the 
complete tables have the properties associated with a group multiplication table 
(see section 28.2) but so do the upper left corners of each table taken on their 
own. The relevant parts are shown in bold in the tables 28.9(a) and (b). 

This observation immediately prompts the notion of a subgroup. A subgroup 
of a group G can be formally defined as any non-empty subset H = {H;} of 
G, the elements of which themselves behave as a group under the same rule of 
combination as applies in G itself. As for all groups, the order of the subgroup is 
equal to the number of elements it contains; we will denote it by h or |H|. 

Any group G contains two trivial subgroups: 


(i) G itself; 
(ii) the set Z consisting of the identity element alone. 


All other subgroups of G are termed proper subgroups. In a group with multipli- 
cation table 28.8 the elements {7, A, B} form a proper subgroup, as do {I, A} ina 
group with table 28.3 as its group table. 

Some groups have no proper subgroups. For example, the so-called cyclic 
groups, mentioned at the end of subsection 28.1.1, have no subgroups other 
than the whole group or the identity alone. Tables 28.10(a) and (b) show the 
multiplication tables for two of these groups. Table 28.6 is also the group table 
for a cyclic group, that of order 4. 
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Sawant 
Soa sare 
ard 
Ber ogqa 
Avan vBIS 


Sawan 


Table 28.10 The group tables of two cyclic groups, of orders 3 and 5. They 
have no proper subgroups. 


It will be clear that for a cyclic group G repeated combination of any element 
with itself generates all other elements of G, before finally reproducing itself. So, 
for example, in table 28.10(b), starting with (say) D, repeated combination with 
itself produces, in turn, C, B, A, I and finally D again. As noted earlier, in any 
cyclic group G every element, apart from the identity, is of order g, the order of 
the group itself. 

The two tables shown are for groups of orders 3 and 5. It will be proved in 
subsection 28.7.2 that the order of any group is a multiple of the order of any of 
its subgroups (Lagrange’s theorem), i.e. in our general notation, g is a multiple 
of h. It thus follows that a group of order p, where p is any prime, must be cyclic 
and cannot have any proper subgroups. The groups for which tables 28.10(a) and 
(b) are the group tables are two such examples. Groups of non-prime order may 
(table 28.3) or may not (table 28.6) have proper subgroups. 

As we have seen, repeated multiplication of an element X (not the identity) 
by itself will generate a subgroup {X, X?, X°,...}. The subgroup will clearly be 
Abelian, and if X is of order m, ic. X™ =I, the subgroup will have m distinct 
members. If m is less than g — though, in view of Lagrange’s theorem, m must 
be a factor of g — the subgroup will be a proper subgroup. We can deduce, in 
passing, that the order of any element of a group is an exact divisor of the order 
of the group. 

Some obvious properties of the subgroups of a group G, which can be listed 
without formal proof, are as follows. 


(i) The identity element of G belongs to every subgroup H. 
(ii) If element X belongs to a subgroup H, so does X—!. 
(iii) The set of elements in G that belong to every subgroup of G themselves 
form a subgroup, though this may consist of the identity alone. 


Properties of subgroups that need more explicit proof are given in the follow- 
ing sections, though some need the development of new concepts before they 
can be established. However, we can begin with a theorem, applicable to all 
homomorphisms, not just isomorphisms, that requires no new concepts. 

Let ® :G — G’ be a homomorphism of G into G’; then 
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(i) the set of elements H’ in G’ that are images of the elements of G forms a 
subgroup of G’; 

(ii) the set of elements K in G that are mapped onto the identity I’ in G’ forms 
a subgroup of G. 


As indicated in the previous section, the subgroup K is called the kernel of the 
homomorphism. 

To prove (i), suppose Z and W belong to H’, with Z = X’ and W = Y’, where 
X and Y belong to G. Then 


ZW =X'Y'=(XY) 
and therefore belongs to H’, and 
Z1 = (xy! = (x-!y! 


and therefore belongs to H’. These two results, together with the fact that I’ 
belongs to H’, are enough to establish result (i). 
To prove (1i), suppose X and Y belong to K; then 


(XY H=XY =1T =T' (closure), 
I’ = (Xx7!y’ — X'(x7ly = I'(x7ty = (x7!) 


and therefore X—! belongs to K. These two results, together with the fact that I 
belongs to K, are enough to establish (ii). An illustration of this result is provided 
by the mapping ® of R — U(1) considered in the previous section. Its kernel 
consists of the set of real numbers of the form 2zn, where n is an integer; it forms 
a subgroup of 7, the additive group of real numbers. 

In fact the kernel K of a homomorphism is a normal subgroup of G. The 
defining property of such a subgroup is that for every element X in G and every 
element Y in the subgroup, XY X~! belongs to the subgroup. This property is 
easily verified for the kernel K, since 


AVX) SR VY Sr eye YS. 


Anticipating the discussion of subsection 28.7.2, the cosets of a normal subgroup 
themselves form a group (see exercise 28.16). 


28.7 Subdividing a group 


We have already noted, when looking at the (arbitrary) order of headings in a 
group table, that some choices appear to make the table more orderly than do 
others. In the following subsections we will identify ways in which the elements 
of a group can be divided up into sets with the property that the members of any 
one set are more like the other members of the set, in some particular regard, 
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than they are like any element that does not belong to the set. We will find that 
these divisions will be such that the group is partitioned, 1.e. the elements will be 
divided into sets in such a way that each element of the group belongs to one, 
and only one, such set. 

We note in passing that the subgroups of a group do not form such a partition, 
not least because the identity element is in every subgroup, rather than being in 
precisely one. In other words, despite the nomenclature, a group is not simply the 
aggregate of its proper subgroups. 


28.7.1 Equivalence relations and classes 


We now specify in a more mathematical manner what it means for two elements 
of a group to be ‘more like’ one another than like a third element, as mentioned 
in section 28.2. Our introduction will apply to any set, whether a group or not, 
but our main interest will ultimately be in two particular applications to groups. 
We start with the formal definition of an equivalence relation. 

An equivalence relation on a set S is a relationship X ~ Y, between two 
elements X and Y belonging to S, in which the definition of the symbol ~ must 
satisfy the requirements of 


(i) reflexivity, ¥ ~ X; 
(ii) symmetry, X¥ ~ Y implies Y ~ X; 
(ili) transitivity, X ~ Y and Y ~ Z imply X ~ Z. 


Any particular two elements either satisfy or do not satisfy the relationship. 

The general notion of an equivalence relation is very straightforward, and 
the requirements on the symbol ~ seem undemanding; but not all relationships 
qualify. As an example within the topic of groups, if it meant ‘has the same 
order as’ then clearly all the requirements would be satisfied. However, if it meant 
‘commutes with’ then it would not be an equivalence relation, since although A 
commutes with J, and I commutes with C, this does not necessarily imply that A 
commutes with C, as is obvious from table 28.8. 

It may be shown that an equivalence relation on S divides up S into classes C; 
such that: 


(i) X and Y belong to the same class if, and only if, X ~ Y; 
(ii) every element W of S belongs to exactly one class. 


This may be shown as follows. Let X belong to S, and define the subset Sy of 
S to be the set of all elements U of S such that X¥ ~ U. Clearly by reflexivity 
X belongs to Sy. Suppose first that X ~ Y, and let Z be any element of Sy. 
Then Y ~ Z, and hence by transitivity X ~ Z, which means that Z belongs to 
Sx. Conversely, since the symmetry law gives Y ~ X, if Z belongs to Sy then 
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this implies that Z belongs to Sy. These two results together mean that the two 
subsets Sy and Sy have the same members and hence are equal. 

Now suppose that Sy equals Sy. Since Y belongs to Sy it also belongs to Sx 
and hence X ~ Y. This completes the proof of (i), once the distinct subsets of 
type Sy are identified as the classes C;. Statement (ii) is an immediate corollary, 
the class in question being identified as Sy. 

The most important property of an equivalence relation is as follows. 


Two different subsets Sy and Sy can have no element in common, and the collection 
of all the classes C; is a ‘partition’ of S, i.e. every element in S belongs to one, and 
only one, of the classes. 


To prove this, suppose Sx and Sy have an element Z in common; then X ~ Z 
and Y ~ Z and so by the symmetry and transitivity laws X ~ Y. By the above 
theorem this implies Sy equals Sy. But this contradicts the fact that Sy and Sy 
are different subsets. Hence Sy and Sy can have no element in common. 

Finally, if the elements of S are used in turn to define subsets and hence classes 
in S, every element U is in the subset Sy that is either a class already found or 
constitutes a new one. It follows that the classes exhaust S, i.e. every element is 
in some class. 

Having established the general properties of equivalence relations, we now turn 
to two specific examples of such relationships, in which the general set S has the 
more specialised properties of a group G and the equivalence relation ~ is chosen 
in such a way that the relatively transparent general results for equivalence 
relations can be used to derive powerful, but less obvious, results about the 
properties of groups. 


28.7.2 Congruence and cosets 


As the first application of equivalence relations we now prove Lagrange’s theorem 
which is stated as follows. 


Lagrange’s theorem. If G is a finite group of order g and H is a subgroup of G of 
order h then g is a multiple of h. 


We take as the definition of ~ that, given X and Y belonging to G, X ~ Y if 
X—'Y belongs to H. This is the same as saying that Y = XH; for some element 
H; belonging to 1; technically X and Y are said to be left-congruent with respect 
to H. 

This defines an equivalence relation, since it has the following properties. 


(i) Reflexivity: X ~ X, since X-!X =I and J belongs to any subgroup. 

(ii) Symmetry: X ~ Y implies that X~!Y belongs to H and so, therefore, does 
its inverse, since H is a group. But (xX! Y)-! = Y~!X and, as this belongs 
to H, it follows that Y ~ xX. 
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(iii) Transitivity: X ~ Y and Y ~ Z imply that X~'Y and Y~'Z belong to H 
and so, therefore, does their product (X~!Y )(Y~!Z) = X—!Z, from which 
it follows that X ~ Z. 


With ~ proved as an equivalence relation, we can immediately deduce that it 
divides G into disjoint (non-overlapping) classes. For this particular equivalence 
relation the classes are called the left cosets of Ht. Thus each element of G is in 
one and only one left coset of 1. The left coset containing any particular X is 
usually written XH, and denotes the set of elements of the form XH; (one of 
which is X itself since #1 contains the identity element); it must contain h different 
elements, since if it did not, and two elements were equal, 


XH; = XH, 


we could deduce that Hj; = H; and that H contained fewer than h elements. 

From our general results about equivalence relations it now follows that the 
left cosets of H are a ‘partition’ of G into a number of sets each containing h 
members. Since there are g members of G and each must be in just one of the 
sets, it follows that g is a multiple of h. This concludes the proof of Lagrange’s 
theorem. 

The number of left cosets of H in G is known as the index of H in G and is 
written [G : 71]; numerically the index = g/h. For the record we note that, for 
the trivial subgroup Z, which contains only the identity element, [G :Z] = g and 
that, for a subgroup J of subgroup H, [G :H][H : 7] =[G: J]. 

The validity of Lagrange’s theorem was established above using the far-reaching 
properties of equivalence relations. However, for this specific purpose there is a 
more direct and self-contained proof, which we now give. 

Let X be some particular element of a finite group G of order g, and H be a 
subgroup of G of order h, with typical element Y;. Consider the set of elements 


XH = {XY1, XYo,...,X Yn}. 


This set contains h distinct elements, since if any two were equal, ie. XY; = XY; 
with i # j, this would contradict the cancellation law. As we have already seen, 
the set is called a left coset of H. 

We now prove three simple results. 


e Two cosets are either disjoint or identical. Suppose cosets X;H and X5H have 
an element in common, ie. X,Y, = XY, for some Yj, Y, in H. Then X;, = 
XN, and since Y; and Y> both belong to H so does YY;'; thus X, 
belongs to the left coset X,H. Similarly X belongs to the left coset X,H. 
Consequently, either the two cosets are identical or it was wrong to assume 
that they have an element in common. 
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e Two cosets X;H and XH are identical if, and only if, Xy'X, belongs to H. If 
Xy'X belongs to H then X; = X2Y; for some i, and 


XH = XiY;H = XoH, 


since by the permutation law Y; = H. Thus the two cosets are identical. 

Conversely, suppose X1;H = X2H. Then Xy'XMH = H. But one element of 
H (on the left of the equation) is I; thus X;!X, must also be an element of H 
(on the right). This proves the stated result. 


e Every element of G is in some left coset XH. This follows trivially since 
contains I, and so the element X; is in the coset X;H. 


The final step in establishing Lagrange’s theorem is, as previously, to note that 
each coset contains h elements, that the cosets are disjoint and that every one of 
the g elements in G appears in one and only one distinct coset. It follows that 
g =kh for some integer k. 

As noted earlier, Lagrange’s theorem justifies our statement that any group of 
order p, where p is prime, must be cyclic and cannot have any proper subgroups: 
since any subgroup must have an order that divides p, this can only be 1 or p, 
corresponding to the two trivial subgroups Z and the whole group. 

It may be helpful to see an example worked through explicitly, and we again 
use the same six-element group. 


> Find the left cosets of the proper subgroup H of the group G that has table 28.8 as its 
multiplication table. 


The subgroup consists of the set of elements H = {I, A,B}. We note in passing that it has 
order 3, which, as required by Lagrange’s theorem, is a divisor of 6, the order of G. As in 
all cases, 1 itself provides the first (left) coset, formally the coset 


TH = {I1,1A,1B} = {1, A, B}. 
We continue by choosing an element not already selected, C say, and form 
CH = {CI,CA,CB} = {C,D,E}. 


These two cosets of # exhaust G, and are therefore the only cosets, the index of H in G 
being equal to 2. 

This completes the example, but it is useful to demonstrate that it would not have 
mattered if we had taken D, say, instead of I to form a first coset 


DH = {DI, DA, DB} = {D,E,C}, 
and then, from previously unselected elements, picked B, say: 
BH = {BI, BA, BB} = {B,1, A}. 
The same two cosets would have resulted. < 
It will be noticed that the cosets are the same groupings of the elements 


of G which we earlier noted as being the choice of adjacent column and row 
headings that give the multiplication table its ‘neatest’ appearance. Furthermore, 
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if H is a normal subgroup of G then its (left) cosets themselves form a group (see 
exercise 28.16). 


28.7.3 Conjugates and classes 


Our second example of an equivalence relation is concerned with those elements 
X and Y of a group G that can be connected by a transformation of the form 
Y = G;'XG;, where G; is an (appropriate) element of G. Thus X ~ Y if there 
exists an element G; of G such that Y = G71 XGi. Different pairs of elements X 
and Y will, in general, require different group elements G;. Elements connected 
in this way are said to be conjugates. 

We first need to establish that this does indeed define an equivalence relation, 
as follows. 


(i) Reflexivity: X ~ X, since X =I~!XI and I belongs to the group. 
(ii) Symmetry: X ~ Y implies Y = G;!XG; and therefore X = (G;')"!Y G71. 
Since G; belongs to G so does GI, and it follows that Y ~ xX. 
(iii) Transitivity: X ~ Y and Y ~ Z imply Y = G;!XG; and Z = G|!YG; 
and therefore Z = G;'G'XGiG; = (GiG;)'X(GiG;). Since G; and G; 
belong to G so does G;G;, from which it follows that X ~ Z. 


These results establish conjugacy as an equivalence relation and hence show 
that it divides G into classes, two elements being in the same class if, and only if, 
they are conjugate. 

Immediate corollaries are: 


(i) If Z is in the class containing I then 


Z=G'1G,=G'G =I. 


Thus, since any conjugate of I can be shown to be J, the identity must be 
in a class by itself. 
(ii) If X is in a class by itself then 


Y =G;!XG; 
must imply that Y = X. But 
X =GiG)'XG,G;! 
for any Gj, and so 
X =G6(G1XG)G;! =GYG;! =G.XxG", 


Le. XG; = G;X for all G;. 

Thus commutation with all elements of the group is a necessary (and 
sufficient) condition for any particular group element to be in a class by 
itself. In an Abelian group each element is in a class by itself. 
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(iii) In any group G the set S of elements in classes by themselves is an Abelian 
subgroup (known as the centre of G). We have shown that I belongs to S, 
and so if, further, XG; = G;X and YG; = G;Y for all G; belonging to G 
then: 
(a) (XY)G; = XG)Y = G,(XY), i.e. the closure of S, and 
(b) XG; = G:X implies X—'G; = G;X—, ie. the inverse of X belongs 
to S. 


Hence S is a group, and clearly Abelian. 


Yet again for illustration purposes, we use the six-element group that has 
table 28.8 as its group table. 


> Find the conjugacy classes of the group G having table 28.8 as its multiplication table. 


As always, I is in a class by itself, and we need consider it no further. 
Consider next the results of forming X~'AX, as X runs through the elements of G. 


I-'AI A7!AA BAB C7!AC D"!AD_ E!AE 
IA IA AI CE DC ED 
A A A B B B 


Only A and B are generated. It is clear that {A, B} is one of the conjugacy classes of G. 
This can be verified by forming all elements X~'BX; again only A and B appear. 

We now need to pick an element not in the two classes already found. Suppose we 
pick C. Just as for A, we compute X~'CX, as X runs through the elements of G. The 
calculations can be done directly using the table and give the following: 


Xx :I A BC DE 
X'CX :C ED C E D 


Thus C, D and E belong to the same class. The group is now exhausted, and so the three 
conjugacy classes are 
{I}, {A, B}, {C,D, E}. < 


In the case of this small and simple, but non-Abelian, group, only the identity 
is in a class by itself (ie. only J commutes with all other elements). It is also the 
only member of the centre of the group. 

Other areas from which examples of conjugacy classes can be taken include 
permutations and rotations. Two permutations can only be (but are not nec- 
essarily) in the same class if their cycle specifications have the same structure. 
For example, in Ss the permutations (1 3 5)(2)(4) and (2 5 3)(1)(4) could be in 
the same class as each other but not in the class that contains (1 5)(2 4)(3). An 
example of permutations with the same cycle structure yet in different conjugacy 
classes is given in exercise 29. 10. 

In the case of the continuous rotation group, rotations by the same angle 0 
about any two axes labelled i and j are in the same class, because the group 
contains a rotation that takes the first axis into the second. Without going into 
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mathematical details, a rotation about axis i can be represented by the operator 


R,(0), and the two rotations are connected by a relationship of the form 


Rj(0) = $7)' Ri(O) Gi, 


in which @j;; is the member of the full continuous rotation group that takes axis 
i into axis j. 


28.1 


28.2 


28.3 


28.8 Exercises 


For each of the following sets, determine whether they form a group under the op- 
eration indicated (where it is relevant you may assume that matrix multiplication 
is associative): 


a) the integers (mod 10) under addition; 

b) the integers (mod 10) under multiplication; 

c) the integers 1,2,3,4,5,6 under multiplication (mod 7); 
) the integers 1,2,3,4,5 under multiplication (mod 6); 
e) all matrices of the form 


anne 
Q. 


where a and b are integers (mod 5) and a #0 # b, under matrix multiplica- 
tion; 

those elements of the set in (e) that are of order 1 or 2 (taken together); 

g) all matrices of the form 


~ 
= 
= 


> 


0 0 
1 0 
c ol 


Sa 


where a, b, c are integers, under matrix multiplication. 


Which of the following relationships between X and Y are equivalence relations? 
Give a proof of your conclusions in each case: 


a) X and Y are integers and X — Y is odd; 

b) X and Y are integers and X — Y is even; 

c) X and Y are people and have the same postcode; 

d) X and Y are people and have a parent in common; 

) X and Y are people and have the same mother; 

f) X and Y arenxn matrices satisfying Y = P XQ, where P and Q are elements 
of a group G of n x n matrices. 


Define a binary operation e on the set of real numbers by 
xey=x+yt+rxy, 


where r is a non-zero real number. Show that the operation e is associative. 

Prove that x e y = —r~! if, and only if, x = —r~! or y = —r7!. Hence prove 
that the set of all real numbers excluding —r~! forms a group under the oper- 
ation e. 
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28.4 


28.5 


28.6 


28.7 


28.8 


28.9 


28.10 


Prove that the relationship X ~ Y, defined by X ~ Y if Y can be expressed in 
the form 

_— aX +b 

~ ¢oX +d’ 
with a, b, c and d as integers, is an equivalence relation on the set of real numbers 
®. Identify the class that contains the real number 1. 
The following is a ‘proof’ that reflexivity is an unnecessary axiom for an equiva- 
lence relation. 


Because of symmetry X ~ Y implies Y ~ X. Then by transitivity X ~ Y and 
Y ~ X imply X ~ X. Thus symmetry and transitivity imply reflexivity, which 
therefore need not be separately required. 


Demonstrate the flaw in this proof using the set consisting of all real numbers plus 
the number i. Show by investigating the following specific cases that, whether or 
not reflexivity actually holds, it cannot be deduced from symmetry and transitivity 
alone. 


(a) X ~ Y if X + Y is real. 
(b) X ~ Y if XY is real. 


Prove that the set M of matrices 


where a, b, c are integers (mod 5) and a # 0 # c, form a non-Abelian group 
under matrix multiplication. 

Show that the subset containing elements of M that are of order 1 or 2 do 
not form a proper subgroup of M, 


(a) using Lagrange’s theorem, 
(b) by direct demonstration that the set is not closed. 


S is the set of all 2 x 2 matrices of the form 


a=(¥ ee where wz — xy = 1. 
yz 
Show that S is a group under matrix multiplication. Which element(s) have order 
2? Prove that an element A has order 3 if w+z+1=0. 

Show that, under matrix multiplication, matrices of the form 


dog + ai —a, + ai 

M(4o, a) = ( a + ai do — ai : 
where do and the components of column matrix a = (a; a 43)! are real 
numbers satisfying aj + |a|’? = 1, constitute a group. Deduce that, under the 
transformation z > Mz, where z is any column matrix, |z|? is invariant. 
If A is a group in which every element other than the identity, J, has order 2, 
prove that A is Abelian. Hence show that if X and Y are distinct elements of A, 
neither being equal to the identity, then the set {[,X,Y, XY} forms a subgroup 
of A. 

Deduce that if B is a group of order 2p, with p a prime greater than 2, then B 
must contain an element of order p. 
The group of rotations (excluding reflections and inversions) in three dimensions 
that take a cube into itself is known as the group 432 (or O in the usual chemical 
notation). Show by each of the following methods that this group has 24 elements. 
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28.11 


28.12 


28.13 


28.14 


28.15 


28.16 


(a) Identify the distinct relevant axes and count the number of qualifying rota- 
tions about each. 

(b) The orientation of the cube is determined if the directions of two of its body 
diagonals are given. Consider the number of distinct ways in which one body 
diagonal can be chosen to be ‘vertical’, say, and a second diagonal made to 
lie along a particular direction. 


Identify the eight symmetry operations on a square. Show that they form a 
group D4, (known to crystallographers as 4mm and to chemists as C4,) having one 
element of order 1, five of order 2 and two of order 4. Find its proper subgroups 
and the corresponding cosets. 
If A and B are two groups, then their direct product, A x B, is defined to be 
the set of ordered pairs (X,Y), with X an element of A, Y an element of B 
and multiplication given by (X, Y)(X’, Y’) = (XX', YY’). Prove that A x Bis a 
group. 

Denote the cyclic group of order n by C,, and the symmetry group of a regular 
n-sided figure (an n-gon) by D,, — thus D3 is the symmetry group of an equilateral 
triangle, as discussed in the text. 


(a) By considering the orders of each of their elements, show (i) that C) x C3 is 
isomorphic to C6, and (ii) that C, x D3 is isomorphic to De. 
(b) Are any of Ds, Cs, C2 x C4, C2 x C2 X Cz isomorphic? 


Find the group G generated under matrix multiplication by the matrices 


0 1 0 i 
a-($ a a= (| a 
Determine its proper subgroups, and verify for each of them that its cosets 
exhaust G. 


Show that if p is prime then the set of rational number pairs (a,b), excluding 
(0,0), with multiplication defined by 


(a,b) e (c,d) =(e,f), where (a+ b,/pyc + d./p) =e + f./p, 


forms an Abelian group. Show further that the mapping (a,b) — (a,—b) is an 

automorphism. 

Consider the following mappings between a permutation group and a cyclic 

group. 

(a) Denote by A, the subset of the permutation group S, that contains all the 
even permutations. Show that A, is a subgroup of S,. 

(b) List the elements of S3 in cycle notation and identify the subgroup A3. 

(c) For each element X of S3, let p(X) = 1 if X belongs to A3 and p(X) = —1 if it 
does not. Denote by C, the multiplicative cyclic group of order 2. Determine 
the images of each of the elements of S3 for the following four mappings: 


®, :83 ~C X > p(X), 
D, y S3 =% Co xXx-> —p(X), 
®3 : 83 > A3 Kine Xt, 
@,4 : S; > S3 Xo Xx}, 
(d) For each mapping, determine whether the kernel K is a subgroup of S3 and, 


if so, whether the mapping is a homomorphism. 


For the group G with multiplication table 28.8 and proper subgroup H = {I, A, B}, 
denote the coset {I,A,B} by C; and the coset {C,D,E} by Cy. Form the set of 
all possible products of a member of C, with itself, and denote this by C,C;. 
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28.17 


28.18 


28.19 


28.20 


28.21 


28.22 


28.23 


Similarly compute C2C2, CiC2 and C2C;. Show that each product coset is equal to 
C; or to Co, and that a 2 x 2 multiplication table can be formed, demonstrating 
that C; and C, are themselves the elements of a group of order 2. A subgroup 
like 7( whose cosets themselves form a group is a normal subgroup. 


The group of all non-singular n x n matrices is known as the general linear 
group GL(n) and that with only real elements as GL(n,R). If R" denotes the 
multiplicative group of non-zero real numbers, prove that the mapping ® : 
GL(n, R) > R*, defined by ®(M) = det M, is a homomorphism. 

Show that the kernel K of ® is a subgroup of GL(n,R). Determine its cosets 
and show that they themselves form a group. 


The group of reflection—rotation symmetries of a square is known as D4; let 
X be one of its elements. Consider a mapping ® : Dy > Sy, the permutation 
group on four objects, defined by ®(X) = the permutation induced by X on 
the set {x,y,d,d'}, where x and y are the two principal axes, and d and d’ 
the two principal diagonals, of the square. For example, if R is a rotation by 
m/2, ®(R) = (12)(34). Show that Dy is mapped onto a subgroup of S4 and, by 
constructing the multiplication tables for D4, and the subgroup, prove that the 
mapping is a homomorphism. 

Given that matrix M is a member of the multiplicative group GL(3, R), determine, 
for each of the following additional constraints on M (applied separately), whether 
the subset satisfying the constraint is a subgroup of GL(3,R): 


(a) MT =M; 
(b) M7M=1; 
(c) |M| = 1; 


(d) Mj; =0 for j >i and M; #0. 
The elements of the quaternion group, Q, are the set 


{1,-1,i,—i, j, —j,k, —k}, 


with ? =? =k? 1, ij =k and its cyclic permutations, and ji = —k and 
its cyclic permutations. Find the proper subgroups of Q and the corresponding 
cosets. Show that the subgroup of order 2 is a normal subgroup, but that the 
other subgroups are not. Show that Q cannot be isomorphic to the group 4mm 
(C4,) considered in exercise 28.11. 

Show that D4, the group of symmetries of a square, has two isomorphic subgroups 
of order 4. Show further that there exists a two-to-one homomorphism from the 
quaternion group Q, of exercise 28.20, onto one (and hence either) of these two 
subgroups, and determine its kernel. 


Show that the matrices 


cos@ —sinO x 
M(0, x, y) = sin@  cosO y |, 
0 0 1 


where 0 < 0 < 22, -~ < x < ~, —~ < y < , form a group under matrix 
multiplication. 


Show that those M(0@,x, y) for which 0 = 0 form a subgroup and identify its 
cosets. Show that the cosets themselves form a group. 


Find (a) all the proper subgroups and (b) all the conjugacy classes of the symmetry 
group of a regular pentagon. 
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28.1 


28.3 


28.5 


28.7 


28.9 


28.11 


28.13 


28.15 


my(t) 


ma(7) 


m3(r) ~ ~ ma(s) 


Figure 28.3 The notation for exercise 28.11. 


28.9 Hints and answers 


S (a) Yes, (b) no, there is no inverse for 2, (c) yes, (d) no, 2 x 3 is not in the set, 
(e) yes, (f) yes, they form a subgroup of order 4, [1,0;0, 1] [4,0;0,4] [1,2;0, 4] 
[4,3:0, 11, (g) yes. 

xe(yez)=x+y+z4+r(xy +xz+ yz) +r?xyz = (xe y)ez. Show that assuming 
xey=-—r! leads to (rx + 1)(ry + 1) = 0. The inverse of x is x~! = —x/(1 +rx); 
show that this is not equal to —r7!. 

(a) Consider both X =i and X #i. Here, i i. (b) In this case i ~ i, but the 
conclusion cannot be deduced from the other axioms. In both cases i is in a class 
by itself and no Y, as used in the false proof, can be found. 

¥ Use |AB| = |A||B| = 1x1 = 1 to prove closure. The inverse has w — z, x < —x, 
y — —y, giving |A~!| = 1, ie. it is in the set. The only element of order 2 is —I; 
A’ can be simplified to [-(w + 1), —x;—y, -(z + 1)]. 

If XY = Z, show that Y = XZ and X = ZY, then form YX. Note that the 
elements of 6 can only have orders 1, 2 or p. Suppose they all have order 1 or 
2; then using the earlier result, whilst noting that 4 does not divide 2p, leads to 
a contradiction. 

Using the notation indicated in figure 28.3, R being a rotation of 2/2 about an 
axis perpendicular to the square, we have: I has order 1; R?, m, m2, m3, m4 have 
order 2; R, R? have order 4. 

subgroup {J, R, R’, R°} has cosets {1,R, R?, R*}, {my,m2,m3, m4}; 

subgroup {I, R?,m,m2} has cosets {I, R*,m,m}, {R, R3, m3, m4} ; 

subgroup {J, R?, m3, m4} has cosets {I, R*,m3,m4}, {R, R3,m), m9}; 

subgroup {J, R?} has cosets {1,R?}, {R, R*}, {m1, 1m}, {m3,ma}; 

subgroup {I,m} has cosets {I,m}, {R,m3}, {R?,m}, {R3,ma}; 

subgroup {I,m} has cosets {I,m}, {R,m4}, {R?,m}, {R3,m3}; 

subgroup {I,m} has cosets {I,m3}, {R,mo}, {R?,m4}, {R3,mi}; 

subgroup {I,m} has cosets {I,m4}, {R,m}, {R?,m3}, {R3,m}. 

G = {I,A,B,B’, B*, AB, AB’, AB}. The proper subgroups are as follows: 

{LA}, {1,B*}, {1,AB*}, {1,B,B’, B*}, {I,B?, AB, AB}. 

(b) As = {(1), (123), (132)}. 

(d) For ©), K = {(1), (123), (132)} is a subgroup. 

For ®, K = {(23), (13), (12)} is not a subgroup because it has no identity element. 
For ®3, K = {(1), (23), (13), (12)} is not a subgroup because it is not closed. 


S Where matrix elements are given as a list, the convention used is [row 1;row 2;...], individual 
entries in each row being separated by commas. 


1074 


28.9 HINTS AND ANSWERS 


28.17 


28.19 
28.21 


28.23 


For ®4, K = {(1), (123), (132)} is a subgroup. 

Only ®; is a homomorphism; 4 fails because, for example, [(23)(13)]’ 4 
(23)'(13Y. 

Recall that, for any pair of matrices P and Q, |PQ| = |P||Q|. K is the set of all 
matrices with unit determinant. The cosets of K are the sets of matrices whose 
determinants are equal; XK itself is the identity in the group of cosets. 

(a) No, because the set is not closed, (b) yes, (c) yes, (d) yes. 

Each subgroup contains the identity, a rotation by z, and two reflections. The 
homomorphism is +1 > 1, +i > R*, +j > my, +k > my with kernel {1,—1}. 
There are 10 elements in all: J, rotations R' (i = 1,4) and reflections m; (j = 1,5). 
(a) There are five proper subgroups of order 2, {I,m,} and one proper subgroup 
of order 5, {I, R, R’, R’, R*}. 

(b) Four conjugacy classes, {I}, {R, R4}, {R?, R?}, {m, mo, m3, m4, ms}. 
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Representation theory 


As indicated at the start of the previous chapter, significant conclusions can 
often be drawn about a physical system simply from the study of its symmetry 
properties. That chapter was devoted to setting up a formal mathematical basis, 
group theory, with which to describe and classify such properties; the current 
chapter shows how to implement the consequences of the resulting classifications 
and obtain concrete physical conclusions about the system under study. The 
connection between the two chapters is akin to that between working with 
coordinate-free vectors, each denoted by a single symbol, and working with a 
coordinate system in which the same vectors are expressed in terms of components. 

The ‘coordinate systems’ that we will choose will be ones that are expressed in 
terms of matrices; it will be clear that ordinary numbers would not be sufficient, 
as they make no provision for any non-commutation amongst the elements 
of a group. Thus, in this chapter the group elements will be represented by 
matrices that have the same commutation relations as the members of the group, 
whatever the group’s original nature (symmetry operations, functional forms, 
matrices, permutations, etc.). For some abstract groups it is difficult to give a 
written description of the elements and their properties without recourse to such 
representations. Most of our applications will be concerned with representations 
of the groups that consist of the symmetry operations on molecules containing 
two or more identical atoms. 

Firstly, in section 29.1, we use an elementary example to demonstrate the kind 
of conclusions that can be reached by arguing purely on symmetry grounds. Then 
in sections 29.2—29.10 we develop the formal side of representation theory and 
establish general procedures and results. Finally, these are used in section 29.11 
to tackle a variety of problems drawn from across the physical sciences. 
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iB 
A 
Fa ee ae ie 
‘4! (apt 
(a) HCI (b) CO, (c) O; 


Figure 29.1 Three molecules, (a) hydrogen chloride, (b) carbon dioxide and 
(c) ozone, for which symmetry considerations impose varying degrees of 
constraint on their possible electric dipole moments. 


29.1 Dipole moments of molecules 


Some simple consequences of symmetry can be demonstrated by considering 
whether a permanent electric dipole moment can exist in any particular molecule; 
three simple molecules, hydrogen chloride, carbon dioxide and ozone, are illus- 
trated in figure 29.1. Even if a molecule is electrically neutral, an electric dipole 
moment will exist in it if the centres of gravity of the positive charges (due to 
protons in the atomic nuclei) and of the negative charges (due to the electrons) 
do not coincide. 

For hydrogen chloride there is no reason why they should coincide; indeed, the 
normal picture of the binding mechanism in this molecule is that the electron from 
the hydrogen atom moves its average position from that of its proton nucleus to 
somewhere between the hydrogen and chlorine nuclei. There is no compensating 
movement of positive charge, and a net dipole moment is to be expected — and 
is found experimentally. 

For the linear molecule carbon dioxide it seems obvious that it cannot have 
a dipole moment, because of its symmetry. Putting this rather more rigorously, 
we note that any rotation about the long axis of the molecule leaves it totally 
unchanged; consequently, any component of a permanent electric dipole perpen- 
dicular to that axis must be zero (a non-zero component would rotate although 
no physical change had taken place in the molecule). That only leaves the pos- 
sibility of a component parallel to the axis. However, a rotation of a radians 
about the axis AA’ shown in figure 29.1(b) carries the molecule into itself, as 
does a reflection in a plane through the carbon atom and perpendicular to the 
molecular axis (1.e. one with its normal parallel to the axis). In both cases the two 
oxygen atoms change places but, as they are identical, the molecule is indistin- 
guishable from the original. Either ‘symmetry operation’ would reverse the sign 
of any dipole component directed parallel to the molecular axis; this can only be 
compatible with the indistinguishability of the original and final systems if the 
parallel component is zero. Thus on symmetry grounds carbon dioxide cannot 
have a permanent electric dipole moment. 
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Finally, for ozone, which is angular rather than linear, symmetry does not 
place such tight constraints. A dipole-moment component parallel to the axis 
BB’ (figure 29.1(c)) is possible, since there is no symmetry operation that reverses 
the component in that direction and at the same time carries the molecule into 
an indistinguishable copy of itself. However, a dipole moment perpendicular to 
BB’ is not possible, since a rotation of z about BB’ would both reverse any 
such component and carry the ozone molecule into itself — two contradictory 
conclusions unless the component is zero. 

In summary, symmetry requirements appear in the form that some or all 
components of permanent electric dipoles in molecules are forbidden; they do 
not show that the other components do exist, only that they may. The greater 
the symmetry of the molecule, the tighter the restrictions on potentially non-zero 
components of its dipole moment. 

In section 23.11 other, more complicated, physical situations will be analysed 
using results derived from representation theory. In anticipation of these results, 
and since it may help the reader to understand where the developments in the 
next nine sections are leading, we make here a broad, powerful, but rather formal, 
statement as follows. 


If a physical system is such that after the application of particular rotations or 
reflections (or a combination of the two) the final system is indistinguishable from 
the original system then its behaviour, and hence the functions that describe its 
behaviour, must have the corresponding property of invariance when subjected to 
the same rotations and reflections. 


29.2 Choosing an appropriate formalism 


As mentioned in the introduction to this chapter, the elements of a finite group 
G can be represented by matrices; this is done in the following way. A suitable 
column matrix u, known as a basis vector’ is chosen and is written in terms of 
its components u;, the basis functions, as u = (wu; U2 --- Un)’. The u; may be of 
a variety of natures, e.g. numbers, coordinates, functions or even a set of labels, 
though for any one basis vector they will all be of the same kind. 

Once chosen, the basis vector can be used to generate an n-dimensional rep- 
resentation of the group as follows. An element X of the group is selected and 
its effect on each basis function u; is determined. If the action of X on uy is to 
produce u', etc. then the set of equations 


u, = Xu; (29.1) 


8 This usage of the term basis vector is not exactly the same as that introduced in subsection 8.1.1. 
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generates a new column matrix u’ = (ui, wu --- u/,)’. Having established u and u’ 
we can determine the n x n matrix, M(X) say, that connects them by 


u’ = M(X)u. (29.2) 


It may seem natural to use the matrix M(X) so generated as the representative 
matrix of the element X; in fact, because we have already chosen the convention 
whereby Z = XY implies that the effect of applying element Z is the same as that 
of first applying Y and then applying X to the result, one further step has to be 
taken. So that the representative matrices D(X) may follow the same convention, 
ie. 


D(Z) = D(X)D(Y), 


and at the same time respect the normal rules of matrix multiplication, it is 
necessary to take the transpose of M(X) as the representative matrix D(X). 
Explicitly, 


D(X) = M1(X) (29.3) 
and (29.2) becomes 
u’ =D'"(X)u. (29.4) 


Thus the procedure for determining the matrix D(X) that represents the group 
element X in a representation based on basis vector u is summarised by equations 
(29.1)-(29.4).$ 

This procedure is then repeated for each element X of the group, and the 
resulting set of n x n matrices D = {D(X)} is said to be the n-dimensional 
representation of G having u as its basis. The need to take the transpose of each 
matrix M(X) is not of any fundamental significance, since the only thing that 
really matters is whether the matrices D(X) have the appropriate multiplication 
properties — and, as defined, they do. 

In cases in which the basis functions are labels, the actions of the group 
elements are such as to cause rearrangements of the labels. Correspondingly the 
matrices D(X) contain only ‘1’s and ‘0’s as entries; each row and each column 
contains a single ‘1’. 


S An alternative procedure in which a row vector is used as the basis vector is possible. Defining 
equations of the form u!X = u'D(X) are used, and no additional transpositions are needed to 
define the representative matrices. However, row-matrix equations are cumbersome to write out 
and in all other parts of this book we have adopted the convention of writing operators (here the 
group element) to the left of the object on which they operate (here the basis vector). 
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>For the group S3; of permutations on three objects, which has group multiplication ta- 
ble 28.8 on p. 1055, with (in cycle notation ) 


1=(1)(2)(3), A=(123), B=(132 
C =(1)(23), D=(3)12), E=(2)(13), 


use as the components of a basis vector the ordered letter triplets 


u, = {PQR}, u. ={QRP}, u3 = {RPQ', 
us = {PRQI, us ={QPR}, up = {RQPI. 


Generate a six-dimensional representation D = {D(X)} of the group and confirm that the 
representative matrices multiply according to table 28.8, e.g. 


D(C)D(B) = D(E). 


It is immediate that the identity permutation I = (1)(2)(3) leaves all u; unchanged, ie. 
u; = u; for all i. The representative matrix D(/) is thus Ic, the 6 x 6 unit matrix. 

We next take X as the permutation A = (123) and, using (29.1), let it act on each of 
the components of the basis vector: 


ul, = Au, = (123{PQR} = {QRP} =u 
us = Aun = (123){QRP} = {RPQ} =u; 
ug = Aus = (123){RQP} = {QPR} = us. 


The matrix M(A) has to be such that u’ = M(A)u (here dots replace zeros to aid readability): 


uz G 1 : ¢ : ; uy 
U3 . 1 uz 
ne uy = Loe oe oe ee U3 _ 
u = ie moa we 6 hi = M(A)u. 
u4 1 G . us 
Us - 1 U6 


D(A) is then equal to M'(A). 
The other D(X) are determined in a similar way. In general, if 


Xuj = Uj, 
then [M(X)],; = 1, leading to [D(X)],; = 1 and [D(X)] ;, = 0 for k # i. For example, 
Cuz = (1)(23){R PQ} = {RQP} = u6 
implies that [D(C)]¢,; = 1 and [D(C)],, = 0 for k = 1,2,4,5,6. When calculated in full 
1. - 1 


D(C) = 
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P P P 
1 

R 2 z Q R : : Q R Q 
(a) (b) (c) 


Figure 29.2 Diagram (a) shows the definition of the basis vector, (b) shows 
the effect of applying a clockwise rotation of 27/3 and (c) shows the effect of 
applying a reflection in the mirror axis through Q. 


from which it can be verified that D(C)D(B) = D(E). < 


Whilst a representation obtained in this way necessarily has the same dimension 
as the order of the group it represents, there are, in general, square matrices of 
both smaller and larger dimensions that can be used to represent the group, 
though their existence may be less obvious. 

One possibility that arises when the group elements are symmetry opera- 
tions on an object whose position and orientation can be referred to a space 
coordinate system is called the natural representation. In it the representative 
matrices D(X) describe, in terms of a fixed coordinate system, what happens 
to a coordinate system that moves with the object when X is applied. There 
is usually some redundancy of the coordinates used in this type of represen- 
tation, since interparticle distances are fixed and fewer than 3N coordinates, 
where N is the number of identical particles, are needed to specify uniquely 
the object’s position and orientation. Subsection 29.11.1 gives an example that 
illustrates both the advantages and disadvantages of the natural representation. 
We continue here with an example of a natural representation that has no such 
redundancy. 


> Use the fact that the group considered in the previous worked example is isomorphic to 
the group of two-dimensional symmetry operations on an equilateral triangle to generate a 


three-dimensional representation of the group. 


Label the triangle’s corners as 1, 2, 3 and three fixed points in space as P, Q, R, so that 
initially corner 1 lies at point P, 2 lies at point Q, and 3 at point R. We take P, Q, R as 
the components of the basis vector. 

In figure 29.2, (a) shows the initial configuration and also, formally, the result of applying 
the identity I to the triangle; it is therefore described by the basis vector, (P Q_ R)!. 

Diagram (b) shows the the effect of a clockwise rotation by 27/3, corresponding to 
element A in the previous example; the new column matrix is(Q R_ P)!. 

Diagram (c) shows the effect of a typical mirror reflection — the one that leaves the 
corner at point Q unchanged (element D in table 28.8 and the previous example); the new 
column matrix isnow(R Q P)!. 

In similar fashion it can be concluded that the column matrix corresponding to element 
B, rotation by 47/3, is (RP Q)', and that the other two reflections C and E result in 
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column matrices(P R Q)' and(Q PR)! respectively. The forms of the representative 
matrices M™'(X), (29.2), are now determined by equations such as, for element E, 


Q 010 P 

Pp |=[1 0 0 Q 

R 001 R 

01 0\" 0 1 0 
poey={ 1 00) =1 100). 

001 001 


In this way the complete representation is obtained as 


implying that 


100 001 0 1 0 
py=f 0 1 0), v4={ 100), p™s=[{[ 0 0 1 |, 
00 1 0 1 0 100 
100 001 0 1 0 
pb““cy={ 0 0 1), v= 0 10), p™ey=[{[ 1 0 0 |. 
0 1 0 100 001 


It should be emphasised that although the group contains six elements this representation 
is three-dimensional. < 


We will concentrate on matrix representations of finite groups, particularly 
rotation and reflection groups (the so-called crystal point groups). The general 
ideas carry over to infinite groups, such as the continuous rotation groups, but in 
a book such as this, which aims to cover many areas of applicable mathematics, 
some topics can only be mentioned and not explored. We now give the formal 
definition of a representation. 


Definition. A representation D = {D(X)} of a group G is an assignment of a non- 
singular square n X n matrix D(X) to each element X belonging to G, such that 


(i) DU) =|), the unit n x n matrix, 
(ii) D(X)D(Y ) = D(X Y) for any two elements X and Y belonging to G, i.e. the 
matrices multiply in the same way as the group elements they represent. 


As mentioned previously, a representation by n x n matrices is said to be an 
n-dimensional representation of G. The dimension n is not to be confused with 
g, the order of the group, which gives the number of matrices needed in the 
representation, though they might not all be different. 

A consequence of the two defining conditions for a representation is that the 
matrix associated with the inverse of X is the inverse of the matrix associated 
with X. This follows immediately from setting Y = X~! in (ii): 


D(X)D(X~!) = D(XX7') = DI) = |; 
hence 


D(X~!) = [D(x)]!. 
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As an example, the four-element Abelian group that consists of the set {1, i, —1,—i} 
under ordinary multiplication has a two-dimensional representation based on the 
column matrix (1 i)!: 


x = (57), co = (9G): 
D(—1) & ace D(-i) = ie ae 


The reader should check that D(i)D(—i) = D(1), D(i)D(i) = D(-1) etc., 1e. that 
the matrices do have exactly the same multiplication properties as the elements 


of the group. Having done so, the reader may also wonder why anybody would 
bother with the representative matrices, when the original elements are so much 
simpler to handle! As we will see later, once some general properties of matrix 
representations have been established, the analysis of large groups, both Abelian 
and non-Abelian, can be reduced to routine, almost cookbook, procedures. 

An n-dimensional representation of G is a homomorphism of G into the set of 
invertible n x n matrices (i.e. n X n matrices that have inverses or, equivalently, 
have non-zero determinants); this set is usually known as the general linear group 
and denoted by GL(n). In general the same matrix may represent more than one 
element of G; if, however, all the matrices representing the elements of G are 
different then the representation is said to be faithful, and the homomorphism 
becomes an isomorphism onto a subgroup of GL(n). 

A trivial but important representation is D(X) = |, for all elements X of G. 
Clearly both of the defining relationships are satisfied, and there is no restriction 
on the value of n. However, such a representation is not a faithful one. 

To sum up, in the context of a rotation-reflection group, the transposes of 
the set of n x n matrices D(X) that make up a representation D may be thought 
of as describing what happens to an n-component basis vector of coordinates, 
(x y  -:-)', or of functions, (Y; 2 ---)', the Y; themselves being functions 
of coordinates, when the group operation X is carried out on each of the 
coordinates or functions. For example, to return to the symmetry operations 
on an equilateral triangle, the clockwise rotation by 27/3, R, carries the three- 
dimensional basis vector (xy z)' into the column matrix 


1 3 
ax + By 
3 1 
7 Oy 
Zz 
whilst the two-dimensional basis vector of functions (7 3z? — r)" is unaltered, 
as neither r nor z is changed by the rotation. The fact that z is unchanged by 


any of the operations of the group shows that the components x, y, z actually 
divide (i.e. are ‘reducible’, to anticipate a more formal description) into two sets: 
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one comprises z, which is unchanged by any of the operations, and the other 
comprises x, y, which change as a pair into linear combinations of themselves. 
This is an important observation to which we return in section 29.4. 


29.3 Equivalent representations 


If D is an n-dimensional representation of a group G, and Q is any fixed invert- 
ible n x n matrix (|Q| # 0), then the set of matrices defined by the similarity 
transformation 


Da(X) = Q7'D(X)Q (29.5) 


also forms a representation Dg of G, said to be equivalent to D. We can see from a 
comparison with the definition in section 29.2 that they do form a representation: 


(i) Da) = Q'DIQ = Q711,Q = I, 
(ii) De(X)DQ(¥Y) = Q7'D(X)QQ7'D(Y )Q = Q7'D(X)D(Y)Q 
=Q'D(XY)Q=DQ(XY). 


Since we can always transform between equivalent representations using a non- 
singular matrix Q, we will consider such representations to be one and the same. 

Despite the similarity of words and manipulations to those of subsection 28.7.1, 
that two representations are equivalent does not constitute an ‘equivalence re- 
lation’ — for example, the reflexive property does not hold for a general fixed 
matrix Q. However, if Q were not fixed, but simply restricted to belonging to 
a set of matrices that themselves form a group, then (29.5) would constitute an 
equivalence relation. 

The general invertible matrix Q that appears in the definition (29.5) of equiv- 
alent matrices describes changes arising from a change in the coordinate system 
(i.e. in the set of basis functions). As before, suppose that the effect of an opera- 
tion X on the basis functions is expressed by the action of M(X) (which is equal 
to D™(X)) on the corresponding basis vector: 


u’ = M(X)u = DT (X)u. (29.6) 
A change of basis would be given by ug = Qu and ug = Qu’, and we may write 
Ug = Qu’ = QM(X)u = QD"(X)Q7!ug. (29.7) 

This is of the same form as (29.6), i.e. 
Ug = D'gr(X)ua, (29.8) 


where Dgr(X) = (Q')-'D(X)Q! is related to D(X) by a similarity transforma- 
tion. Thus Dgr(X) represents the same linear transformation as D(X), but with 
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respect to a new basis vector Ug; this supports our contention that representa- 
tions connected by similarity transformations should be considered as the same 
representation. 


>For the four-element Abelian group consisting of the set {1,i,—1,—i} under ordinary 
multiplication, discussed near the end of section 29.2, change the basis vector from u = 
(1 i)’ to ug =(3—i 2i—5)!. Find the real transformation matrix Q. Show that the 
transformed representative matrix for element i, Dar(i), is given by 


; i7eeane 
Dart) = ( gay ) 


and verify that DGr(iua = iug. 


Firstly, we solve the matrix equation 
3-i \ fa b 1 
2i-5 }]  \ cd i}? 
with a,b,c,d real. This gives Q and hence Q7! as 
7 3. -1 sfe,f 2 
ae ca 
Following (29.7) we now find the transpose of Dgr(i) as 
3. -1 0 1 2 1 17 10 
TiyQ-1 — = 
on -( 3, 2 ee ake: 3)-(4% tt) 
and hence Dgr(i) is as stated. Finally, 
er ae 17 10 3-i \_ 1+3i 
Regie = ( H99 ap Paes = boos 
_.f 3-i = 
=F\ o7—5 | = "a 


Ww 
Wa a 


as required. < 


Although we will not prove it, it can be shown that any finite representation 
of a finite group of linear transformations that preserve spatial length (or, in 
quantum mechanics, preserve the magnitude of a wavefunction) is equivalent to 
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a representation in which all the matrices are unitary (see chapter 8) and so from 
now on we will consider only unitary representations. 


29.4 Reducibility of a representation 


We have seen already that it is possible to have more than one representation 
of any particular group. For example, the group {1,i,—1,—i} under ordinary 
multiplication has been shown to have a set of 2 x 2 matrices, and a set of four 
unit n x n matrices |,, as two of its possible representations. 

Consider two or more representations, D!), p@,...,D™), which may be 
of different dimensions, of a group G. Now combine the matrices D(X), 
D(X), ...,D™)(X) that correspond to element X of G into a larger block- 
diagonal matrix: 


D(X) = | Dex) | (29.9) 


0 


Then D = {D(X)} is the matrix representation of the group obtained by combining 
the basis vectors of D"), D®, ..., D“) into one larger basis vector. If, knowingly or 
unknowingly, we had started with this larger basis vector and found the matrices 
of the representation D to have the form shown in (29.9), or to have a form 
that can be transformed into this by a similarity transformation (29.5) (using, 
of course, the same matrix Q for each of the matrices D(X)) then we would say 
that D is reducible and that each matrix D(X) can be written as the direct sum of 
smaller representations: 


D(X) = D(X) @ D(X) @---@ D(X). 


It may be that some or all of the matrices D(X), D?(X), ...,D“) themselves 
can be further reduced — ie. written in block diagonal form. For example, 
suppose that the representation D", say, has a basis vector (xy z)!; then, for 
the symmetry group of an equilateral triangle, whilst x and y are mixed together 
for at least one of the operations X, z is never changed. In this case the 3 x 3 
representative matrix D(X) can itself be written in block diagonal form as a 
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2 x 2 matrix and a 1 x 1 matrix. The direct-sum matrix D(X) can now be written 


0 


D(X) = (29.10) 


but the first two blocks can be reduced no further. 

When all the other representations D(X), ... have been similarly treated, 
what remains is said to be irreducible and has the characteristic of being block 
diagonal, with blocks that individually cannot be reduced further. The blocks are 
known as the irreducible ic oe of G, often abbreviated to the irreps of 
G, and we denote them by 6” . They form the building blocks of representation 
theory, and it is their properties that are used to analyse any given physical 
situation which is invariant under the operations that form the elements of G. 
Any representation can be written as a linear combination of irreps. 

If, however, the initial choice u of basis vector for the representation D is 
arbitrary, as it is in general, then it is unlikely that the matrices D(X) will 
assume obviously block diagonal forms (it should be noted, though, that since 
the matrices are square, even a matrix with non-zero entries only in the extreme 
top right and bottom left positions is technically block diagonal). In general, it 
will be possible to reduce them to block diagonal matrices with more than one 
block; this reduction corresponds to a transformation Q to a new basis vector 
Ug, as described in section 29.3. 

In any particular representation D, each constituent irrep 6” may appear any 
number of times, or not at all, subject to the obvious restriction that the sum of 
all the irrep dimensions must add up to the dimension of D itself. Let us say that 
6” appears m; times. The general expansion of D is then written 


D= m0"? ® mb” @®::-® ins (29.11) 


where if G is finite so is N. 
This is such an important result that we shall now restate the situation in 
somewhat different language. When the set of matrices that forms a representation 
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of a particular group of symmetry operations has been brought to irreducible 
form, the implications are as follows. 


(i) Those components of the basis vector that correspond to rows in the 
representation matrices with a single-entry block, ic. a 1 x 1 block, are 
unchanged by the operations of the group. Such a coordinate or function 
is said to transform according to a one-dimensional irrep of G. In the 
example given in (29.10), that the entry on the third row forms a 1 x 1 
block implies that the third entry in the basis vector (x y z --:)!, 
namely z, is invariant under the two-dimensional symmetry operations on 
an equilateral triangle in the xy-plane. 


(ii) If, in any of the g matrices of the representation, the largest-sized block 
located on the row or column corresponding to a particular coordinate 
(or function) in the basis vector is n x n, then that coordinate (or function) 
is mixed by the symmetry operations with n — 1 others and is said to 
transform according to an n-dimensional irrep of G. Thus in the matrix 
(29.10), x is the first entry in the complete basis vector; the first row of 
the matrix contains two non-zero entries, as does the first column, and so 
x is part of a two-component basis vector whose components are mixed 
by the symmetry operations of G. The other component is y. 


The result (29.11) may also be formulated in terms of the more abstract notion 
of vector spaces (chapter 8). The set of g matrices that forms an n-dimensional 
representation D of the group G can be thought of as acting on column matrices 
corresponding to vectors in an n-dimensional vector space V spanned by the basis 
functions of the representation. If there exists a proper subspace W of V, such 
that if a vector whose column matrix is w belongs to W then the vector whose 
column matrix is D(X)w also belongs to W, for all X belonging to G, then it 
follows that D is reducible. We say that the subspace W is invariant under the 
actions of the elements of G. With D unitary, the orthogonal complement W, of 
W, i.e. the vector space V remaining when the subspace W has been removed, is 
also invariant, and all the matrices D(X) split into two blocks acting separately 
on W and W,. Both W and W, may contain further invariant subspaces, in 
which case the matrices will be split still further. 

As a concrete example of this approach, consider in plane polar coordinates 
p, @ the effect of rotations about the polar axis on the infinite-dimensional vector 
space V of all functions of @ that satisfy the Dirichlet conditions for expansion 
as a Fourier series (see section 12.1). We take as our basis functions the set 
{sin m@, cos m@} for integer values m = 0,1,2,...; this is an infinite-dimensional 
representation (n = oo) and, since a rotation about the polar axis can be through 
any angle « (0 < a < 2z), the group G is a subgroup of the continuous rotation 
group and has its order g formally equal to infinity. 
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Now, for some k, consider a vector w in the space W; spanned by {sink¢, cosk¢}, 
say w = asinkd + bcosk@. Under a rotation by « about the polar axis, asinkd 
becomes asink(¢+«), which can be written as acoskasinkd +asinkacosk4@, i.e 
as a linear combination of sinkd and cosk@; similarly cosk@ becomes another 
linear combination of the same two functions. The newly generated vector w’, 
whose column matrix w is given by w’ = D(a)w, therefore belongs to W; for 
any « and we can conclude that W;, is an invariant irreducible two-dimensional 
subspace of V. It follows that D(a) is reducible and that, since the result holds 
for every k, in its reduced form D(a) has an infinite series of identical 2 x 2 blocks 
on its leading diagonal; each block will have the form 

cosa —sina 
( sing cose ) , 
We note that the particular case k = 0 is special, in that then sink¢ = 0 and 
coskd = 1, for all #; consequently the first 2 x 2 block in D(«) is reducible further 
and becomes two single-entry blocks. 

A second illustration of the connection between the behaviour of vector spaces 
under the actions of the elements of a group and the form of the matrix repre- 
sentation of the group is provided by the vector space spanned by the spherical 
harmonics Y,,,(0,@). This contains subspaces, corresponding to the different 
values of 7, that are invariant under the actions of the elements of the full three- 
dimensional rotation group; the corresponding matrices are block-diagonal, and 
those entries that correspond to the part of the basis containing Y;,,(0,b) form a 
(2¢ + 1) x (2¢ + 1) block. 

To illustrate further the irreps of a group, we return again to the group G of 
two-dimensional rotation and reflection symmetries of an equilateral triangle, or 
equivalently the permutation group S3; this may be shown, using the methods of 
section 29.7 below, to have three irreps. Firstly, we have already seen that the set 
M of six orthogonal 2 x 2 matrices given in section (28.3), equation (28.13), is 
isomorphic to G. These matrices therefore form not only a representation of G, 
but a faithful one. It should be noticed that, although G contains six elements, 
the matrices are only 2 x 2. However, they contain no invariant 1 x 1 sub-block 
(which for 2 x 2 matrices would require them all to be diagonal) and neither can 
all the matrices be made block-diagonal by the same similarity transformation; 
they therefore form a two-dimensional irrep of G. 

Secondly, as previously noted, every group has one (unfaithful) irrep in which 
every element is represented by the 1 x 1 matrix I;, or, more simply, 1. 

Thirdly an (unfaithful) irrep of G is given by assignment of the one-dimensional 
set of six ‘matrices’ {1,1,1,—1,—1,—1} to the symmetry operations {1, R, R’,K, 
L,M} respectively, or to the group elements {1,A,B,C,D,E} respectively; see 
section 28.3. In terms of the permutation group $3, 1 corresponds to even 
permutations and —1 to odd permutations, ‘odd’ or ‘even’ referring to the number 
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of simple pair interchanges to which a permutation is equivalent. That these 
assignments are in accord with the group multiplication table 28.8 should be 
checked. 

Thus the three irreps of the group G (i.e. the group 3m or C3, or S3), are, using 
the conventional notation A;, A», E (see section 29.8), as follows: 


I 
Ai | 1 1 1 1 1 1 (29.12) 
Irrep A. | 1 


where 


29.5 The orthogonality theorem for irreducible representations 


We come now to the central theorem of representation theory, a theorem that 
justifies the relatively routine application of certain procedures to determine 
the restrictions that are inherent in physical systems that have some degree of 
rotational or reflection symmetry. The development of the theorem is long and 
quite complex when presented in its entirety, and the reader will have to refer 
elsewhere for the proof.§ 

The theorem states that, in a certain sense, the irreps of a group G are as 
orthogonal as possible, as follows. If, for each irrep, the elements in any one 
position in each of the g matrices are used to make up g-component column 
matrices then 


(i) any two such column matrices coming from different irreps are orthogonal; 
(ii) any two such column matrices coming from different positions in the 
matrices of the same irrep are orthogonal. 


This orthogonality is in addition to the irreps’ being in the form of orthogo- 
nal (unitary) matrices and thus each comprising mutually orthogonal rows and 
columns. 


S See, e.g., H. F. Jones, Groups, Representations and Physics (Bristol: Institute of Physics, 1998); J. 
F. Cornwell, Group Theory in Physics, vol 2 (London: Academic Press, 1984); J-P. Serre, Linear 
Representations of Finite Groups (New York: Springer, 1977). 
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More mathematically, if we denote the entry in the ith row and jth column of a 
matrix D(X) by [D(X)];;, and 6” and 6” are two irreps of G having dimensions 
n, and n, respectively, then 


ype w), [BO] = Sond dz. (29.13) 
: nj 


This rather forbidding-looking equation needs some further explanation. 

Firstly, the asterisk indicates that the complex conjugate should be taken if 
necessary, though all our representations so far have involved only real matrix 
elements. Each Kronecker delta function on the right-hand side has the value 1 
if its two subscripts are equal and has the value 0 otherwise. Thus the right-hand 
side is only non-zero if i=k, j =/ and A = y, all at the same time. 

Secondly, the summation over the group elements X means that g contributions 
have to be added together, each contribution being a producl: of entries drawn 
from the representative matrices in the two irreps p= = { 6 6.4 } and p= 


A(x). The g contributions arise as X runs over the g elements of G. 
Thus, putting these remarks together, the summation will produce zero if either 


(i) the matrix elements are not taken from exactly the same position in every 
matrix, reuse aad in which it is not possible to do so because the 
irreps A’ |and pe ‘have different dimensions, or 

(ii) even if pe and ah do have the same dimensions and the matrix elements 
are from the same positions in every matrix, they are different irreps, i.e. 


AF UL. 
Some numerical illustrations based on the irreps Ai, Az and E of the group 3m 


(or C3, or S3) will probably provide the clearest explanation (see (29.12)). 


(a) Take i= j =k =! =1, with B® = A, and 6” = Ay, Equation (29.13) 
then reads 


1(1) + 1(1) + 11) + 1(—1) + 1(—1) + 1(—-1) = 0, 
as expected, since A # uu. 
(b) Take (i,j) as (1,2) and (k,/) as (2, 2), con eHondine to different matrix 


positions within the same irrep P= BO Sk. Substituting in (29.13) 

gives 

o(1) + (—38)(-4) + ()(-4) +001) + (—B)(-4) + ()(-4) = 0. 
(c) Take (i,j) as (1,2), and (k,1) as (1,2), corresponding to the same matrix 


positions within the same irrep D Do =D" =e Substituting in (29.13) 


v0 (-)(8) +H) +004 88) +H) <8 
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(d) No explicit calculation is needed to see that if i = j =k =1 = 1, with 
Sheen ee A, (or A»), then each term in the sum is either 1? or (—1)? 
and the total is 6, as predicted by the right-hand side of (29.13) since g = 6 


and n, = 1. 


29.6 Characters 


The actual matrices of general representations and irreps are cumbersome to 
work with, and they are not unique since there is always the freedom to change 
the coordinate system, i.e. the components of the basis vector (see section 29.3), 
and hence the entries in the matrices. However, one thing that does not change 
for a matrix under such an equivalence (similarity) transformation — i.e. under 
a change of basis — is the trace of the matrix. This was shown in chapter 8, 
but is repeated here. The trace of a matrix A is the sum of its diagonal ele- 
ments, 


n 
TrA= DS Aii 
i=1 


or, using the summation convention (section 26.1), simply Aj. Under a similarity 
transformation, again using the summation convention, 

[Da(X)]ii = [Q7']: [D(X] jx [Qi 
D(X)] x [Q]ciTO~" 
D(X)] ix Way 
D(X)] jj, 


ll 


ll 


ll 


[ 
[ 
[ 
[ 


ll 


showing that the traces of equivalent matrices are equal. 

This fact can be used to greatly simplify work with representations, though with 
some partial loss of the information content of the full matrices. For example, 
using trace values alone it is not possible to distinguish between the two groups 
known as 4mm and 42m, or as C4, and Dg respectively, even though the two 
groups are not isomorphic. To make use of these simplifications we now define 
the characters of a representation. 


Definition. The characters y(D) of a representation D of a group G are defined as 
the traces of the matrices D(X), one for each element X of G. 


At this stage there will be g characters, but, as we noted in subsection 28.7.3, 
elements A, B of G in the same conjugacy class are connected by equations of 
the form B = X—'AX. It follows that their matrix representations are connected 
by corresponding equations of the form D(B) = D(X—'!)D(A)D(X), and so by the 
argument just given their representations will have equal traces and hence equal 
characters. Thus elements in the same conjugacy class have the same characters, 
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z327; x? +y? 
R, 
(x, y)3 (82,2); (Re Ry); (x? — y?, 2xy) 


Table 29.1 The character table for the irreps of group 3m (C3, or S3). The 
right-hand column lists some common functions that transform according to 
the irrep against which each is shown (see text). 


though, in general, these will vary from one representation to another. However, 
it might also happen that two or more conjugacy classes have the same characters 
in a representation — indeed, in the trivial irrep Aj, see (29.12), every element 
inevitably has the character 1. 

For the irrep A> of the group 3m, the classes {1}, {A,B} and {C,D,E} have 
characters 1, 1 and —1, respectively, whilst they have characters 2, —1 and 0 
respectively in irrep E. 

We are thus able to draw up a character table for the group 3m as shown 
in table 29.1. This table holds in compact form most of the important infor- 
mation on the behaviour of functions under the two-dimensional rotational and 
reflection symmetries of an equilateral triangle, ic. under the elements of group 
3m. The entry under J for any irrep gives the dimension of the irrep, since it 
is equal to the trace of the unit matrix whose dimension is equal to that of 
the irrep. In other words, for the Ath irrep y(I) = nj, where n, is its dimen- 
sion. 

In the extreme right-hand column we list some common functions of Cartesian 
coordinates that transform, under the group 3m, according to the irrep on whose 
line they are listed. Thus, as we have seen, z, z*, and x? + y? are all unchanged 
by the group operations (though x and y individually are affected) and so are 
listed against the one-dimensional irrep A;. Each of the pairs (x, y), (xz, yz), and 
(x? — y?, 2xy), however, is mixed as a pair by some of the operations, and so these 
pairs are listed against the two-dimensional irrep E: each pair forms a basis set 
for this irrep. 

The quantities R,, R, and R, refer to rotations about the indicated axes; 
they transform in the same way as the corresponding components of angular 
momentum J, and their behaviour can be established by examining how the 
components of J = r x p transform under the operations of the group. To do 
this explicitly is beyond the scope of this book. However, it can be noted that 
R,, being listed opposite the one-dimensional A), is unchanged by J and by the 
rotations A and B but changes sign under the mirror reflections C, D, and E, as 
would be expected. 
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29.6.1 Orthogonality property of characters 


Some of the most important properties of characters can be deduced from the 
orthogonality theorem (29.13), 


a [Bo], [B”” wo], 2 56 18% 


If we set j = i and | =k, so that both factors in any particular term in the 
summation refer to diagonal elements of the representative matrices, and then 
sum both sides over i and k, we obtain 


ny My ny My 


TLD. [BPE]. [BP CO}, = ESO Vdd din 


X i=l k=1 MT kel 


Expressed in term of characters, this reads 


nj 


Se 00) 0X) = > ya re ao X Say = BO 


x (29.14) 


In words, the (g-component) ‘vectors’ formed from the characters of the various 
irreps of a group are mutually orthogonal, but each one has a squared magnitude 
(the sum of the squares of its components) equal to the order of the group. 
Since, as noted in the previous subsection, group elements in the same class 
have the same characters, (29.14) can be written as a sum over classes rather than 
elements. If c; denotes the number of elements in class C; and X; any element of 
C;, then 
Soci [XRD] (Xi) = 85ip. (29.15) 
T 
Although we do not prove it here, there also exists a ‘completeness’ relation for 
characters. It makes a statement about the products of characters for a fixed pair 
of group elements, X,; and X>, when the products are summed over all possible 


irreps of the group. This is the converse of the summation process defined by 
(29.14). The completeness relation states that 


Se [OR] xX) = = Secs (29.16) 
2 
where element X; belongs to conjugacy class C; and X> belongs to Cz. Thus the 


sum is zero unless X; and X, belong to the same class. For table 29.1 we can 
verify that these results are valid. 


(i) For pp = A, or Ag, (29.15) reads 
1(1) + 2(1) + 3(1) = 
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whilst for 6” = 6 = E, it gives 
1(27) + 2(1) + 3(0) = 6. 
(li) For A” = A> and p= E, say, (29.15) reads 
1(1)(2) + 2(1)(—1) + 3(—1)(0) = 0. 
(iii) For X; = A and X2 = D, say, (29.16) reads 
1(1) + 1(-1) + (-1)(0) = 0, 


whilst for X; = C and X2 = E, both of which belong to class C3 for which 
B= 3, 


1(1) + (—1)(-1) + (0)(0) = 2 = -. 


29.7 Counting irreps using characters 


The expression of a general representation D = {D(X)} in terms of irreps, as 
given in (29.11), can be simplified by going from the full matrix form to that of 
characters. Thus 


A(1 A (2 A(N 
D(X) = m8 (X) @ mb" (X) @---@ myD(X) 


becomes, on taking the trace of both sides, 


N 
MX) = So may (X). (29.17) 
j=1 
Given the characters of the irreps of the group G to which the elements X belong, 
and the characters of the representation D = {D(X)}, the g equations (29.17) 
can be solved as simultaneous equations in the m;, either by inspection or by 
multiplying both sides by n(x)" and summing over X, making use of (29.14) 
and (29.15), to obtain 


My = -> [eX] xX) = -> ci [x"(Xi)]" (Xi). (29.18) 
x i 

That an unambiguous formula can be given for each m,, once the character 
set (the set of characters of each of the group elements or, equivalently, of 
each of the conjugacy classes) of D is known, shows that, for any particular 
group, two representations with the same characters are equivalent. This strongly 
suggests something that can be shown, namely, the number of irreps = the number 
of conjugacy classes. The argument is as follows. Equation (29.17) is a set of 
simultaneous equations for N unknowns, the m,, some of which may be zero. The 
value of N is equal to the number of irreps of G. There are g different values of 
X, but the number of different equations is only equal to the number of distinct 
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conjugacy classes, since any two elements of G in the same class have the same 
character set and therefore generate the same equation. For a unique solution 
to simultaneous equations in N unknowns, exactly N independent equations are 
needed. Thus N is also the number of classes, establishing the stated result. 


> Determine the irreps contained in the representation of the group 3m in the vector space 
spanned by the functions x”, y*, xy. 


We first note that although these functions are not orthogonal they form a basis set for a 
representation, since they are linearly independent quadratic forms in x and y and any other 
quadratic form can be written (uniquely) in terms of them. We must establish how they 
transform under the symmetry operations of group 3m. We need to do so only for a repre- 
sentative element of each conjugacy class, and naturally we take the simplest in each case. 
The first class contains only I (as always) and clearly D(/) is the 3 x 3 unit matrix. 
The second class contains the rotations, A and B, and we choose to find D(A). Since, 


under A, 
3 id 
eee er and Pe ig ig Vs 
it follows that 
xo 4X? Bxy +3y’, ya xt Bxy +49” (29.19) 
and 
sy. pay? Sty a a8, (29.20) 


Hence D(A) can be deduced and is given below. 

The third and final class contains the reflections, C, D and E; of these C is much the 
easiest to deal with. Under C, x + —x and y — jy, causing xy to change sign but leaving 
x? and y* unaltered. The three matrices needed are thus 


7 
10 0 a, a eS 

DU)=h, DIC)={ 0 1 0 |, DA=] 3 2  ; 
0 0 -1 J 8 _1 
a 4 2 


their traces are respectively 3, 1 and 0. 

It should be noticed that much more work has been done here than is necessary, since 
the traces can be computed immediately from the effects of the symmetry operations on the 
basis functions. All that is needed is the weight of each basis function in the transformed 
expression for that function; these are clearly 1, 1, 1 for I, and +, 4, —4 for A, from (29.19) 
and (29.20), and 1, 1, —1 for C, from the observations made just above the displayed 
matrices. The traces are then the sums of these weights. The off-diagonal elements of the 
matrices need not be found, nor need the matrices be written out. 

From (29.17) we now need to find a superposition of the characters of the irreps that 
gives representation D in the bottom line of table 29.2. 

By inspection it is obvious that D = A; @ E, but we can use (29.18) formally: 


ma, = ¢[1(1)(3) + 2(1)(0) + 3()(1)] = 1, 
ma, = 4[1(1)(3) + 2(1)(0) + 3(—1)(1)] = 0, 
me = 4[1(2)(3) + 2(—1)(0) + 3(0)(1)] = 1. 


Thus A; and E appear once each in the reduction of D, and A» not at all. Table 29.1 
gives the further information, not needed here, that it is the combination x? + y? that 
transforms as a one-dimensional irrep and the pair (x? — y?, 2xy) that forms a basis of 
the two-dimensional irrep, E. < 
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Classes 
AB CDE 
1 1 
1 —1 
—1 0 
0 1 


Table 29.2 The characters of the irreps of the group 3m and of the represen- 
tation D, which must be a superposition of some of them. 


29.7.1 Summation rules for irreps 


The first summation rule for irreps is a simple restatement of (29.14), with yu set 
equal to A; it then reads 


3 eo] Ww) = 8. 
x 


In words, the sum of the squares (modulus squared if necessary) of the characters 
of an irrep taken over all elements of the group adds up to the order of the 
group. For group 3m (table 29.1), this takes the following explicit forms: 


for Ay, 1(17) + 2(17) + 3(17) = 6; 
for Ao, 1(1?) + 2(17) + 3(-1) = 6; 
for E, 1(27) + 2(-1)? + 3(0”) = 6. 


We next prove a theorem that is concerned not with a summation within an irrep 
but with a summation over irreps. 


Theorem. If n, is the dimension of the uth irrep of a group G then 
2 
ye My = 8 
u 
where g is the order of the group. 


Proof. Define a representation of the group in the following way. Rearrange 
the rows of the multiplication table of the group so that whilst the elements in 
a particular order head the columns, their inverses in the same order head the 
rows. In this arrangement of the g x g table, the leading diagonal is entirely 
occupied by the identity element. Then, for each element X of the group, take as 
representative matrix the multiplication-table array obtained by replacing X by 
1 and all other element symbols by 0. The matrices D'*(X) so obtained form the 
regular representation of G; they are each g x g, have a single non-zero entry ‘1’ 
in each row and column and (as will be verified by a little experimentation) have 


1097 


REPRESENTATION THEORY 


(b) 


~& apa 
ar wld 


wat |— 
mt 

moh |— 
ae ES 
~aw|o 


Table 29.3 (a) The multiplication table of the cyclic group of order 3, and 
(b) its reordering used to generate the regular representation of the group. 


the same multiplication structure as the group G itself, ie. they form a faithful 
representation of G. 

Although not part of the proof, a simple example may help to make these 
ideas more transparent. Consider the cyclic group of order 3. Its multiplication 
table is shown in table 29.3(a) (a repeat of table 28.10(a) of the previous chapter), 
whilst table 29.3(b) shows the same table reordered so that the columns are 
still labelled in the order J, A, B but the rows are now labelled in the order 
I-!' =], A-'=B, B-! =A. The three matrices of the regular representation are 
then 


10 0 01 0 Go a 
per)={| 0 1 0 |, p™(4)=[{[ 0 0 1 |, DeB)=| 1 0 0 
001 1020 0 1 0 


An alternative, more mathematical, definition of the regular representation of a 
group is 


1 if G.G;=G, 


0 otherwise. 


[D'**(G,)] iz 


We now return to the proof. With the construction given, the regular representa- 
tion has characters as follows: 


C8Q=g, 7¢°(X)=0 if X £1. 


We now apply (29.18) to D'® to obtain for the number m, of times that the irrep 


p appears in D’® (see 29.11)) 


1 ‘ 1 * 1 
my = eos [MOO] 72x) = 5 MO) x8) = ns = Me 
xX 


Thus an irrep 6” of dimension n, appears n, times in D’®, and so by counting 
the total number of basis functions, or by considering y8(I), we can conclude 
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that 
Sim=g. (29.21) 
Lu 


This completes the proof. 

As before, our standard demonstration group 3m provides an illustration. In 
this case we have seen already that there are two one-dimensional irreps and one 
two-dimensional irrep. This is in accord with (29.21) since 


1?+1°+4+2?=6, which is the order g of the group. 


Another straightforward application of the relation (29.21), to the group with 
multiplication table 29.3(a), yields immediate results. Since g = 3, none of its 
irreps can have dimension 2 or more, as 27 = 4 is too large for (29.21) to be 
satisfied. Thus all irreps must be one-dimensional and there must be three of 
them (consistent with the fact that each element is in a class of its own, and that 
there are therefore three classes). The three irreps are the sets of 1 x 1 matrices 
(numbers) 


A; = {1,1,13 A> = {1,@,@7} A} = {1,07, 0}, 


where w = exp(27i/3); since the matrices are 1 x 1, the same set of nine numbers 
would be, of course, the entries in the character table for the irreps of the group. 
The fact that the numbers in each irrep are all cube roots of unity is discussed 
below. As will be noticed, two of these irreps are complex — an unusual occurrence 
in most applications — and form a complex conjugate pair of one-dimensional 
irreps. In practice, they function much as a two-dimensional irrep, but this is to 
be ignored for formal purposes such as theorems. 

A further property of characters can be derived from the fact that all elements 
in a conjugacy class have the same order. Suppose that the element X has order 
m, Le. X™ =I. This implies for a representation D of dimension n that 


[D(X)]”" = Ih. (29.22) 


Representations equivalent to D are generated as before by using similarity 
transformations of the form 


De(X) = Q7'!D(X)Q. 


In particular, if we choose the columns of Q to be the eigenvectors of D(X) then, 
as discussed in chapter 8, 


dn 70 0 

0 A 

Da(X) = ‘ 
a 
0 0 An 
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where the /; are the eigenvalues of D(X). Therefore, from (29.22), we have that 


At 0 ++ 0 | a 
Os Ay Bs, ita Ot 
: - 0 : we 

Oo -. 0 am Oo -..- 0 1 


Hence all the eigenvalues 4; are mth roots of unity, and so 7(X), the trace of 
D(X), is the sum of n of these. In view of the implications of Lagrange’s theorem 
(section 28.6 and subsection 28.7.2), the only values of m allowed are the divisors 
of the order g of the group. 


29.8 Construction of a character table 


In order to decompose representations into irreps on a routine basis using 
characters, it is necessary to have available a character table for the group in 
question. Such a table gives, for each irrep u of the group, the character y)(X) 
of the class to which group element X belongs. To construct such a table the 
following properties of a group, established earlier in this chapter, may be used: 


(i) the number of classes equals the number of irreps; 

(ii) the ‘vector’ formed by the characters from a given irrep is orthogonal to 
the ‘vector’ formed by the characters from a different irrep; 

(iii) D7, nm, = g, where n, is the dimension of the pth irrep and g is the order 
of the group; 

(iv) the identity irrep (one-dimensional with all characters equal to 1) is present 
for every group; 

(v) Dy PCO) = 8. 

(vi) p(X) is the sum of n, mth roots of unity, where m is the order of X. 


> Construct the character table for the group 4mm (or C4, ) using the properties of classes, 
irreps and characters so far established. 


The group 4mm is the group of two-dimensional symmetries of a square, namely rotations 
of 0, 2/2, x and 32/2 and reflections in the mirror planes parallel to the coordinate axes 
and along the main diagonals. These are illustrated in figure 29.3. For this group there are 
eight elements: 


e the identity, I; 

e rotations by 2/2 and 32/2, R and R’; 

e a rotation by z, Q ; 

e four mirror reflections my, my, mg and my. 

Requirements (i) to (iv) at the start of this section put tight constraints on the possible 
character sets, as the following argument shows. 


The group is non-Abelian (clearly Rm, # m,R), and so there are fewer than eight 
classes, and hence fewer than eight irreps. But requirement (iii), with g = 8, then implies 
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Sele eee Asin xo -------J---- m, 


Figure 29.3 The mirror planes associated with 4mm, the group of two- 
dimensional symmetries of a square. 


that at least one irrep has dimension 2 or greater. However, there can be no irrep with 
dimension 3 or greater, since 3 > 8, nor can there be more than one two-dimensional 
irrep, since 27 + 2? = 8 would rule out a contribution to the sum in (iii) of 1* from the 
identity irrep, and this must be present. Thus the only possibility is one two-dimensional 
irrep and, to make the sum in (iii) correct, four one-dimensional irreps. 

Therefore using (i) we can now deduce that there are five classes. This same conclusion 
can be reached by evaluating X—!Y X for every pair of elements in G, as in the description 
of conjugacy classes given in the previous chapter. However, it is tedious to do so and 
certainly much longer than the above. The five classes are J, Q, {R, R’}, {m,, my}, {ma, ma}. 

It is straightforward to show that only I and Q commute with every element of the 
group, so they are the only elements in classes of their own. Each other class must have 
at least 2 members, but, as there are three classes to accommodate 8 — 2 = 6 elements, 
there must be exactly 2 in each class. This does not pair up the remaining 6 elements, but 
does say that the five classes have 1, 1, 2, 2, and 2 elements. Of course, if we had started 
by dividing the group into classes, we would know the number of elements in each class 
directly. 

We cannot entirely ignore the group structure (though it sometimes happens that the 
results are independent of the group structure — for example, all non-Abelian groups of 
order 8 have the same character table!); thus we need to note in the present case that 
m? =I fori=x, y, d ord’ and, as can be proved directly, Rm; = m;R’ for the same four 
values of label i. We also recall that for any pair of elements X and Y, D(X Y) = D(X)D(Y). 
We may conclude the following for the one-dimensional irreps. 


(a) In view of result (vi), y(m;) = D(m;) = +1. 
(b) Since R* = IJ, result (vi) requires that 7(R) is one of 1, i, —1, —i. But, since 
D(R)D(m;) = D(m;)D(R’), and the D(m;) are just numbers, D(R) = D(R’). Further 
D(R)D(R) = D(R)D(R’) = D(RR’) = DT) = 1, 
and so D(R) = +1 = D(R’). 
(c) D(Q) = D(RR) = D(R)D(R) = 1. 
If we add this to the fact that the characters of the identity irrep A; are all unity then we 
can fill in those entries in character table 29.4 shown in bold. 


Suppose now that the three missing entries in a one-dimensional irrep are p, q and r, 
where each can only be +1. Then, allowing for the numbers in each class, orthogonality 
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I Q RR mm, mame 
1 1 1 1 1 
1 1 1 —1 —1 
1 1 —1 1 -1 
1 1 —1 —1 1 
2 -2 0 0 0 


Table 29.4 The character table deduced for the group 4mm. For an explana- 
tion of the entries in bold see the text. 


with the characters of A; requires that 
1(1)(1) + 10) 0) + 20) @) + 20)(q) + 20) (7) = 0. 


The only possibility is that two of p, g, and r equal —1 and the other equals +1. This 
can be achieved in three different ways, corresponding to the need to find three further 
different one-dimensional irreps. Thus the first four lines of entries in character table 29.4 
can be completed. The final line can be completed by requiring it to be orthogonal to the 
other four. Property (v) has not been used here though it could have replaced part of the 
argument given. < 


29.9 Group nomenclature 


The nomenclature of published character tables, as we have said before, is erratic 
and sometimes unfortunate; for example, often E is used to represent, not only 
a two-dimensional irrep, but also the identity operation, where we have used I. 
Thus the symbol E might appear in both the column and row headings of a 
table, though with quite different meanings in the two cases. In this book we use 
roman capitals to denote irreps. 

One-dimensional irreps are regularly denoted by A and B, B being used if a 
rotation about the principal axis of 27/n has character —1. Here n is the highest 
integer such that a rotation of 2z/n is a symmetry operation of the system, and 
the principal axis is the one about which this occurs. For the group of operations 
on a square, n = 4, the axis is the perpendicular to the square and the rotation 
in question is R. The names for the group, 4mm and C4,, derive from the fact 
that here n is equal to 4. Similarly, for the operations on an equilateral triangle, 
n = 3 and the group names are 3m and C3,, but because the rotation by 27/3 has 
character +1 in all its one-dimensional irreps (see table 29.1), only A appears in 
the irrep list. 

Two-dimensional irreps are denoted by E, as we have already noted, and three- 
dimensional irreps by T, although in many cases the symbols are modified by 
primes and other alphabetic labels to denote variations in behaviour from one 
irrep to another in respect of mirror reflections and parity inversions. In the study 
of molecules, alternative names based on molecular angular momentum properties 
are common. It is beyond the scope of this book to list all these variations, or to 
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give a large selection of character tables; our aim is to demonstrate and justify 
the use of those found in the literature specifically dedicated to crystal physics or 
molecular chemistry. 

Variations in notation are not restricted to the naming of groups and their 
irreps, but extend to the symbols used to identify a typical element, and hence 
all members, of a conjugacy class in a group. In physics these are usually of the 
types n., 7, or my. The first of these denotes a rotation of 2z/n about the z-axis, 
and the second the same thing followed by parity inversion (all vectors r go to 
—r), whilst the third indicates a mirror reflection in a plane, in this case the plane 
x=0. 

Typical chemistry symbols for classes are NC,, NC?, NCX, NS,, o,, 6%”. Here 
the first symbol N, where it appears, shows that there are N elements in the 
class (a useful feature). The subscript n has the same meaning as in the physics 
notation, but o rather than m is used for a mirror reflection, subscripts v, d or h or 
superscripts xy, xz or yz denoting the various orientations of the relevant mirror 
planes. Symmetries involving parity inversions are denoted by S; thus S, is the 
chemistry analogue of 7. None of what is said in this and the previous paragraph 
should be taken as definitive, but merely as a warning of common variations in 
nomenclature and as an initial guide to corresponding entities. Before using any 
set of group character tables, the reader should ensure that he or she understands 
the precise notation being employed. 


29.10 Product representations 


In quantum mechanical investigations we are often faced with the calculation of 
what are called matrix elements. These normally take the form of integrals over all 
space of the product of two or more functions whose analytic forms depend on the 
microscopic properties (usually angular momentum and its components) of the 
electrons or nuclei involved. For ‘bonding’ calculations involving ‘overlap integrals’ 
there are usually two functions involved, whilst for transition probabilities a third 
function, giving the spatial variation of the interaction Hamiltonian, also appears 
under the integral sign. 

If the environment of the microscopic system under investigation has some 
symmetry properties, then sometimes these can be used to establish, without 
detailed evaluation, that the multiple integral must have zero value. We now 
express the essential content of these ideas in group theoretical language. 

Suppose we are given an integral of the form 


p= [ ¥oa or p= [ veod 


to be evaluated over all space in a situation in which the physical system is 
invariant under a particular group G of symmetry operations. For the integral to 
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be non-zero the integrand must be invariant under each of these operations. In 
group theoretical language, the integrand must transform as the identity, the one- 
dimensional representation A, of G; more accurately, some non-vanishing part of 
the integrand must do so. 

An alternative way of saying this is that if under the symmetry operations 
of G the integrand transforms according to a representation D and D does not 
contain A; amongst its irreps then the integral J is necessarily zero. It should be 
noted that the converse is not true; J may be zero even if A, is present, since the 
integral, whilst showing the required invariance, may still have the value zero. 

It is evident that we need to establish how to find the irreps that go to make 
up a representation of a double or triple product when we already know the 
irreps according to which the factors in the product transform. The method is 
established by the following theorem. 


Theorem. For each element of a group the character in a product representation is 
the product of the corresponding characters in the separate representations. 


Proof. Suppose that {u;} and {v;} are two sets of basis functions, that transform 
under the operations of a group G according to representations D” and D” 
respectively. Denote by u and v the corresponding basis vectors and let X be an 
element of the group. Then the functions generated from u; and v; by the action 
of X are calculated as follows, using (29.1) and (29.4): 


uf, = Xuj = [(BMX)" uy] = [DPC] 4 + [(0%E0)"] a, 
[Hi 

v =Xvj= [(oen)"y], =) [DX] ,, vj +S° [cowen)"] i re 
m#j 


Here [D(X)];; is just a single element of the matrix D(X) and [D(X)]kx = [DT (XJ ex 
is simply a diagonal element from the matrix — the repeated subscript does not 
indicate summation. Now, if we take as basis functions for a product represen- 
tation DP'4(X) the products w, = ujv; (where the njn, various possible pairs of 
values i, j are labelled by k), we have also that 


wy = Xwe = Xujvj = (Xuj)(Xv;) 
= [D(X)] , [D OO. .ujv; + terms not involving the product ujv;. 
This is to be compared with 
wy = Xwy = [(or™29)"w = [DP4(X)] we + > [(or'e0)"] De 
n 
n#k 


where D?"°4(X) is the product representation matrix for element X of the group. 
The comparison shows that 


[pP"(x)| kk [D”(x)] ii [DY(X)] ,;- 
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It follows that 


nN, 


prod) = a [DPr4(x)] ‘ 
k=1 


ny, Mw 


=>] [p(x)],, [> (X)] 


1=1.j=1 


Z >» [D(x] : 1 pean} 


i=l j=l 
=X PR). (29.23) 


This proves the theorem, and a similar argument leads to the corresponding result 
for integrands in the form of a product of three or more factors. 

An immediate corollary is that an integral whose integrand is the product of 
two functions transforming according to two different irreps is necessarily zero. To 
see this, we use (29.18) to determine whether irrep A; appears in the product 
character set y?4(X): 


1 


ma, = = >> [XPD]! P(X) = ‘yy; yP"4(X) = oe (X)y7(X). 
8g x 8g x 


We have used the fact that 7‘“(X) = 1 for all X but now note that, by virtue of 
(29.14), the expression on the right of this equation is equal to zero unless A = yu. 

Any complications due to non-real characters have been ignored — in practice, 
they are handled automatically as it is usually ¥"¢, rather than V¢, that appears 
in integrands, though many functions are real in any case, and nearly all characters 
are. 

Equation (29.23) is a general result for integrands but, specifically in the context 
of chemical bonding, it implies that for the possibility of bonding to exist, the 
two quantum wavefunctions must transform according to the same irrep. This is 
discussed further in the next section. 


29.11 Physical applications of group theory 


As we indicated at the start of chapter 28 and discussed in a little more detail at 
the beginning of the present chapter, some physical systems possess symmetries 
that allow the results of the present chapter to be used in their analysis. We 
consider now some of the more common sorts of problem in which these results 
find ready application. 
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y, 


Figure 29.4 A molecule consisting of four atoms of iodine and one of 
manganese. 


29.11.1 Bonding in molecules 


We have just seen that whether chemical bonding can take place in a molecule 
is strongly dependent upon whether the wavefunctions of the two atoms forming 
a bond transform according to the same irrep. Thus it is sometimes useful to be 
able to find a wavefunction that does transform according to a particular irrep 
of a group of transformations. This can be done if the characters of the irrep are 
known and a sensible starting point can be guessed. We state without proof that 


starting from any n-dimensional basis vector ¥ = (Y; ‘Yo --- W,,)', where {P;} 
is a set of wavefunctions, the new vector ¥ = (PO pY) ... YH)T generated 
by 
Se Oe ee 8). 6 (29.24) 
x 


will transform according to the Ath irrep. If the randomly chosen Y’ happens not 
to contain any component that transforms in the desired way then the P“ so 
generated is found to be a zero vector and it is necessary to select a new starting 
vector. An illustration of the use of this ‘projection operator’ is given in the next 
example. 


> Consider a molecule made up of four iodine atoms lying at the corners of a square in the 
xy-plane, with a manganese atom at its centre, as shown in figure 29.4. Investigate whether 
the molecular orbital given by the superposition of p-state (angular momentum | = 1) 
atomic orbitals 


VY, = Py —R,) + ¥.(r— Ro) — P5(r — R3) — P(r — Rg) 


can bond to the d-state atomic orbitals of the manganese atom described by either (i) ¢; = 
(32?—r*)f(r) or (ii) 2 = (x?—y*)f(r), where f(r) is a function of r and so is unchanged by 
any of the symmetry operations of the molecule. Such linear combinations of atomic orbitals 
are known as ring orbitals. 


We have eight basis functions, the atomic orbitals ‘¥,(N) and ‘¥,(N), where N = 1, 2, 3, 4 
and indicates the position of an iodine atom. Since the wavefunctions are those of p-states 
they have the forms xf(r) or yf(r) and lie in the directions of the x- and y-axes shown in 
the figure. Since r is not changed by any of the symmetry operations, f(r) can be treated as 
a constant. The symmetry group of the system is 4mm, whose character table is table 29.4. 
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Case (i). The manganese atomic orbital ¢; = (327 — r)f(r), lying at the centre of the 
molecule, is not affected by any of the symmetry operations since z and r are unchanged 
by them. It clearly transforms according to the identity irrep A;. We therefore need to 
know which combination of the iodine orbitals ¥,(N) and Y,(N), if any, also transforms 
according to Ay. 

We use the projection operator (29.24). If we choose ¥,(1) as the arbitrary one- 
dimensional starting vector, we unfortunately obtain zero (as the reader may wish to 
verify), but Y,,(1) is found to generate a new non-zero one-dimensional vector transforming 
according to A;. The results of acting on ¥,(1) with the various symmetry elements X 
can be written down by inspection (see the discussion in section 29.2). So, for example, the 
,(1) orbital centred on iodine atom 1 and aligned along the positive y-axis is changed 
by the anticlockwise rotation of 2/2 produced by R’ into an orbital centred on atom 4 
and aligned along the negative x-axis; thus R’Y,(1) = —‘Y,(4). The complete set of group 
actions on P,(1) is: 


T, y(t); Q, —¥5(3); OR, PAZ); RY, SPA); 
my, ‘Py(1); my, —‘Py(3); ma, ‘Px(2); ma, —‘P x). 


Now 740(X) = 1 for all X, so (29.24) states that the sum of the above results for XP,(1), 
all with weight 1, gives a vector (here, since the irrep is one-dimensional, just a wave- 
function) that transforms according to A, and is therefore capable of forming a chemical 
bond with the manganese wavefunction @y. It is 


PAD = 2[Py(1) — P,(3) + Px(2) — Px(4)], 


hough, of course, the factor 2 is irrelevant. This is precisely the ring orbital ‘¥; given in 
he problem, but here it is generated rather than guessed beforehand. 
Case (ii). The atomic orbital 6) = (x* — y”)f(r) behaves as follows under the action of 
ypical conjugacy class members: 
I, oo; Q, bo; R (Y —X*)f(r) =— 2; my, 2s Mas — 2. 
From this we see that ¢@2 transforms as a one-dimensional irrep, but, from table 29.4, that 


irrep is B; not A; (the irrep according to which ‘P; transforms, as already shown). Thus 
2 and ‘¥; cannot form a bond. < 


The original question did not ask for the the ring orbital to which $2 may 
bond, but it can be generated easily by using the values of X‘¥,(1) calculated in 
case (i) and now weighting them according to the characters of B,: 

HP) = W(1) — By (3) + (1x2) — (x4) 
+ Py(1) — Py(3) + (DP (2) — (—1) (4) 
= 2[P,(1) — Px(2) — ‘Py(3) + P(4)]- 


Now we will find the other irreps of 4mm present in the space spanned by 
the basis functions ‘P,(N) and Y,(N); at the same time this will illustrate the 
important point that since we are working with characters we are only interested 
in the diagonal elements of the representative matrices. This means (section 29.2) 
that if we work in the natural representation D™' we need consider only those 
functions that transform, wholly or partially, into themselves. Since we have no 
need to write out the matrices explicitly, their size (8 x 8) is no drawback. All the 
irreps spanned by the basis functions ‘,(N) and Y,(N) can be determined by 
considering the actions of the group elements upon them, as follows. 
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(i) Under I all eight basis functions are unchanged, and y(I) = 8. 

(ii) The rotations R, R’ and Q change the value of N in every case and so 
all diagonal elements of the natural representation are zero and y(R) = 
x(Q) = 0. 

(ili) m, takes x into —x and y into y and, for N = 1 and 3, leaves N unchanged, 
with the consequences (remember the forms of ‘P,(N) and ,(N)) that 


¥x(1) zh —Y(1), Y(3) > —Y¥ (3), 
¥,(1) — Y,(1), (3) > P,(3). 


Thus y(m,) has four non-zero contributions, —1, —1, 1 and 1, together 
with four zero contributions. The total is thus zero. 
(iv) mg and my leave no atom unchanged and so y(m,) = 0. 


The character set of the natural representation is thus 8, 0, 0, 0, 0, which, either 
by inspection or by applying formula (29.18), shows that 


p™' — A, ®@A, @B, @ By @ 2E, 


ie. that all possible irreps are present. We have constructed previously the 
combinations of ‘¥,(N) and ¥,(N) that transform according to A; and Bj. 
The others can be found in the same way. 


29.11.2 Matrix elements in quantum mechanics 


In section 29.10 we outlined the procedure for determining whether a matrix 
element that involves the product of three factors as an integrand is necessarily 
zero. We now illustrate this with a specific worked example. 


> Determine whether a ‘dipole’ matrix element of the form 


J= [tarts dt, 


where Wy, and ‘Vy, are d-state wavefunctions of the forms xyf(r) and (x? — y”)g(r) respec- 
tively, can be non-zero (i) in a molecule with symmetry C3, (or 3m), such as ammonia, and 
(i) in a molecule with symmetry C4, (or 4mm), such as the MnI, molecule considered in 
the previous example. 


We will need to make reference to the character tables of the two groups. The table for 
C3, is table 29.1 (section 29.6); that for C4, is reproduced as table 29.5 from table 29.4 but 
with the addition of another column showing how some common functions transform. 

We make use of (29.23), extended to the product of three functions. No attention need 
be paid to f(r) and g(r) as they are unaffected by the group operations. 

Case (i). From the character table 29.1 for C3,, we see that each of xy, x and x2 — y? 
forms part of a basis set transforming according to the two-dimensional irrep E. Thus we 
may fill in the array of characters (using chemical notation for the classes, except that 
we continue to use J rather than E) as shown in table 29.6. The last line is obtained by 
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2 
iy? 
xy 
(x, ¥)3 (x2, yz); (Rx, Ry) 


Table 29.5 The character table for the irreps of group 4mm (or C4,). The 
right-hand column lists some common functions, or, for the two-dimensional 
irrep E, pairs of functions, that transform according to the irrep against which 
they are shown. 


Function — Irrep Classes 

xy E 2 -1 0 
x E 2 -1 0 
x? — y? E 2 -1 O 
product 8 -1 0 


Table 29.6 The character sets, for the group C3, (or 3mm), of three functions 
and of their product x”y(x? — y’). 


Function — Irrep Classes 

I Cy 2C6 20, 204 
xy Bo if 1 -1 —1 a 
x E 2 —2 0 0 0 
x? — y? B, 1 1 -1 1 -1 
product 2 -2 0 0 0 


Table 29.7 The character sets, for the group C4, (or 4mm), of three functions, 
and of their product x?y(x? — y’). 


multiplying together the corresponding characters for each of the three elements. Now, by 
inspection, or by applying (29.18), i.e. 

ma, = ZU(D(8) + 2(-L) + 3(DO)] = 1 
we see that irrep A; does appear in the reduced representation of the product, and so J 
is not necessarily zero. 

Case (ii). From table 29.5 we find that, under the group C4,, xy and x? — y* transform 
as irreps By and B, respectively and that x is part of a basis set transforming as E. Thus 
the calculation table takes the form of table 29.7 (again, chemical notation for the classes 
has been used). 

Here inspection is sufficient, as the product is exactly that of irrep E and irrep A; is 
certainly not present. Thus J is necessarily zero and the dipole matrix element vanishes. < 


1109 


REPRESENTATION THEORY 


¥3 


x3 


yi 


a 
x2 
—>X1 


Figure 29.5 An equilateral array of masses and springs. 


29.11.3 Degeneracy of normal modes 


As our final area for illustrating the usefulness of group theoretical results we 
consider the normal modes of a vibrating system (see chapter 9). This analysis 
has far-reaching applications in physics, chemistry and engineering. For a given 
system, normal modes that are related by some symmetry operation have the same 
frequency of vibration; the modes are said to be degenerate. It can be shown that 
such modes span a vector space that transforms according to some irrep of the 
group G of symmetry operations of the system. Moreover, the degeneracy of 
the modes equals the dimension of the irrep. As an illustration, we consider the 
following example. 


> Investigate the possible vibrational modes of the equilateral triangular arrangement of| 
equal masses and springs shown in figure 29.5. Demonstrate that two are degenerate. 


Clearly the symmetry group is that of the symmetry operations on an equilateral triangle, 
namely 3m (or C3,), whose character table is table 29.1. As on a previous occasion, it is 
most convenient to use the natural representation D"' of this group (it almost always 
saves having to write out matrices explicitly) acting on the six-dimensional vector space 
(X1, V1, X2, Y2, X3, 3). In this example the natural and regular representations coincide, but 
this is not usually the case. 

We note that in table 29.1 the second class contains the rotations A (by 2/3) and B (by 
22/3), also known as R and R’. This class is known as 3, in crystallographic notation, or 
C3 in chemical notation, as explained in section 29.9. The third class contains C, D, E, the 
three mirror reflections. 

Clearly y(1) = 6. Since all position labels are changed by a rotation, ¥(3,) = 0. For the 
mirror reflections the simplest representative class member to choose is the reflection m, in 
the plane containing the y3-axis, since then only label 3 is unchanged; under m,, x3 > —Xx3 
and y3 — y3, leading to the conclusion that y(m,) = 0. Thus the character set is 6, 0, 0. 

Using (29.18) and the character table 29.1 shows that 


D™' = A; @ A) @ 2E. 
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However, we have so far allowed x;, yj to be completely general, and we must now identify 
and remove those irreps that do not correspond to vibrations. These will be the irreps 
corresponding to bodily translations of the triangle and to its rotation without relative 
motion of the three masses. 

Bodily translations are linear motions of the centre of mass, which has coordinates 


X = (x1 +X2+43)/3 and y =(y1 + y2 + y3)/3). 


Table 29.1 shows that such a coordinate pair (x,y) transforms according to the two- 
dimensional irrep E; this accounts for one of the two such irreps found in the natural 
representation. 

It can be shown that, as stated in table 29.1, planar bodily rotations of the triangle — 
rotations about the z-axis, denoted by R-, — transform as irrep Az. Thus, when the linear 
motions of the centre of mass, and pure rotation about it, are removed from our reduced 
representation, we are left with E®A;. So, E and A; must be the irreps corresponding to the 
internal vibrations of the triangle — one doubly degenerate mode and one non-degenerate 
mode. 

The physical interpretation of this is that two of the normal modes of the system have 
the same frequency and one normal mode has a different frequency (barring accidental 
coincidences for other reasons). It may be noted that in quantum mechanics the energy 
quantum of a normal mode is proportional to its frequency. < 


In general, group theory does not tell us what the frequencies are, since it is 
entirely concerned with the symmetry of the system and not with the values of 
masses and spring constants. However, using this type of reasoning, the results 
from representation theory can be used to predict the degeneracies of atomic 
energy levels and, given a perturbation whose Hamiltonian (energy operator) has 
some degree of symmetry, the extent to which the perturbation will resolve the 
degeneracy. Some of these ideas are explored a little further in the next section 
and in the exercises. 


29.11.4 Breaking of degeneracies 


If a physical system has a high degree of symmetry, invariant under a group G of 
reflections and rotations, say, then, as implied above, it will normally be the case 
that some of its eigenvalues (of energy, frequency, angular momentum etc.) are 
degenerate. However, if a perturbation that is invariant only under the operations 
of the elements of a smaller symmetry group (a subgroup of G) is added, some of 
the original degeneracies may be broken. The results derived from representation 
theory can be used to decide the extent of the degeneracy-breaking. 

The normal procedure is to use an N-dimensional basis vector, consisting of the 
N degenerate eigenfunctions, to generate an N-dimensional representation of the 
symmetry group of the perturbation. This representation is then decomposed into 
irreps. In general, eigenfunctions that transform according to different irreps no 
longer share the same frequency of vibration. We illustrate this with the following 
example. 
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Figure 29.6 A circular drumskin loaded with three symmetrically placed 
masses. 


>A circular drumskin has three equal masses placed on it at the vertices of an equilateral 
triangle, as shown in figure 29.6. Determine which degenerate normal modes of the drumskin 
can be split in frequency by this perturbation. 


When no masses are present the normal modes of the drum-skin are either non-degenerate 
or two-fold degenerate (see chapter 21). The degenerate eigenfunctions of the nth normal 
mode have the forms 


J,(kr)(cosn0)e = or J, (kr)(sin nO)et!”". 


Therefore, as explained above, we need to consider the two-dimensional vector space 
spanned by ‘¥; = sinn@ and ‘¥, = cosn@. This will generate a two-dimensional representa- 
tion of the group 3m (or C3,), the symmetry group of the perturbation. Taking the easiest 
element from each of the three classes (identity, rotations, and reflections) of group 3m, 
we have 


IW, ct ae I'W, =P), 


AY, = sin [n (0 =- 2n)| = (cos nm) . a (sin nm) Wo, 
AW, = cos [n (0 — $n)] = (cos nm) ¥ + (sin $n) Wy, 
CY, = sin[n(z — 0)] = —(cosnz)¥, 


CW, = cos[n(z — 0)] = (cosnz)'P. 


The three representative matrices are therefore 


cos nn —sin int —cosnt 0 
DU)=b, D(A) = : , DC)= , 


sin $nm — COs nn 0 cos nt 


The characters of this representation are y(I) = 2, x(A) = 2cos(2nz/3) and y(C) = 0. 
Using (29.18) and table 29.1, we find that 


ma, = ¢(2+4cos 3nz) = ma, 


ME = z (4—4cos inn). 


Thus 
p= A, @A, ifn=3, 6,9, ..., 
~ )E otherwise. 
Hence the normal modes n = 3, 6, 9, ... each transform under the operations of 3m 
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as the sum of two one-dimensional irreps and, using the reasoning given in the previous 
example, are therefore split in frequency by the perturbation. For other values of n the 
representation is irreducible and so the degeneracy cannot be split. < 


29.1 


29.2 


29.3 


29.4 


29.12 Exercises 


A group G has four elements I, X,Y and Z, which satisfy X*? = Y* = Z? = 
XYZ =I. Show that G is Abelian and hence deduce the form of its character 
table. 

Show that the matrices 


ou =( 4 i nx) =( Ae 
or=( fee nz) =( 4 fh.) 


where p is a real number, form a representation D of G. Find its characters and 
decompose it into irreps. 

Using a square whose corners lie at coordinates (+1,+1), form a natural rep- 
resentation of the dihedral group D4. Find the characters of the representation, 
and, using the information (and class order) in table 29.4 (p. 1102), express the 
representation in terms of irreps. 

Now form a representation in terms of eight 2 x 2 orthogonal matrices, by 
considering the effect of each of the elements of D4 on a general vector (x, y). 
Confirm that this representation is one of the irreps found using the natural 
representation. 

The quaternion group Q (see exercise 28.20) has eight elements {+1, +i, +j, +k} 
obeying the relations 


P=fP=k? 1, ij=k ji. 
Determine the conjugacy classes of Q and deduce the dimensions of its irreps. 
Show that Q is homomorphic to the four-element group VY, which is generated by 
two distinct elements a and b with a? = b* = (ab)? = I. Find the one-dimensional 
irreps of V and use these to help determine the full character table for Q. 


Construct the character table for the irreps of the permutation group S4 as 
follows. 


(a) By considering the possible forms of its cycle notation, determine the number 
of elements in each conjugacy class of the permutation group S4, and show 
that S4 has five irreps. Give the logical reasoning that shows they must consist 
of two three-dimensional, one two-dimensional, and two one-dimensional 
irreps. 

(b) By considering the odd and even permutations in the group S4, establish the 
characters for one of the one-dimensional irreps. 

(c) Form a natural matrix representation of 4 x 4 matrices based on a set of 
objects {a,b,c,d}, which may or may not be equal to each other, and, by 
selecting one example from each conjugacy class, show that this natural 
representation has characters 4, 2, 1, 0, 0. In the four-dimensional vector 
space in which each of the four coordinates takes on one of the four values 
a, b, c or d, the one-dimensional subspace consisting of the four points with 
coordinates of the form {a,a,a,a} is invariant under the permutation group 
and hence transforms according to the invariant irrep A;. The remaining 
three-dimensional subspace is irreducible; use this and the characters deduced 
above to establish the characters for one of the three-dimensional irreps, T,. 


1113 


REPRESENTATION THEORY 


29.5 


29.6 


29.7 


(d) Complete the character table using orthogonality properties, and check the 
summation rule for each irrep. You should obtain table 29.8. 


Typical element and class size 


(1) (12) (123) (1234) -—-(12)(34) 
1 6 8 6 3 
T 1 I 1 I 
{ <1 1 =i 1 
2 0 -1 0 2 
3 1 0 =i -1 
3 =f 0 1 -1 


Table 29.8 The character table for the permutation group Sy. 


In exercise 28.10, the group of pure rotations taking a cube into itself was found 
to have 24 elements. The group is isomorphic to the permutation group Sy, 
considered in the previous question, and hence has the same character table, once 
corresponding classes have been established. By counting the number of elements 
in each class, make the correspondences below (the final two cannot be decided 
purely by counting, and should be taken as given). 


Permutation Symbol Action 
class type (physics) 


(1) I none 

(123) 3 rotations about a body diagonal 

(12)(34) 2, rotation of xz about the normal to a face 
(1234) 4. rotations of +2/2 about the normal to a face 
(12) 24 rotation of z about an axis through the 


centres of opposite edges 


Reformulate the character table 29.8 in terms of the elements of the rotation 
symmetry group (432 or O) of a cube and use it when answering exercises 29.7 
and 29.8. 

Consider a regular hexagon orientated so that two of its vertices lie on the x-axis. 
Find matrix representations of a rotation R through 27/6 and a reflection m, in 
the y-axis by determining their effects on vectors lying in the xy-plane . Show 
that a reflection m, in the x-axis can be written as m, = myR?, and that the 12 
elements of the symmetry group of the hexagon are given by R" or R"my. 

Using the representations of R and m, as generators, find a two-dimensional 

representation of the symmetry group, C,, of the regular hexagon. Is it a faithful 
representation? 
In a certain crystalline compound, a thorium atom lies at the centre of a regular 
octahedron of six sulphur atoms at positions (+a, 0,0), (0, +a, 0), (0,0, a). These 
can be considered as being positioned at the centres of the faces of a cube of 
side 2a. The sulphur atoms produce at the site of the thorium atom an electric 
field that has the same symmetry group as a cube (432 or O). 

The five degenerate d-electron orbitals of the thorium atom can be expressed, 
relative to any arbitrary polar axis, as 


(3cos?0—1)f(r), e** sin@cosOf(r), e+”? sin? Of (r). 
A rotation about that polar axis by an angle @’ effectively changes ¢ to ¢ — @’. 
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Use this to show that the character of the rotation in a representation based on 
the orbital wavefunctions is given by 


1+ 2cos ¢’ + 2cos 2¢’ 


and hence that the characters of the representation, in the order of the symbols 
given in exercise 29.5, is 5, —1, 1, —1, 1. Deduce that the five-fold degenerate 
level is split into two levels, a doublet and a triplet. 

Sulphur hexafluoride is a molecule with the same structure as the crystalline 
compound in exercise 29.7, except that a sulphur atom is now the central atom. 
The following are the forms of some of the electronic orbitals of the sulphur 
atom, together with the irreps according to which they transform under the 
symmetry group 432 (or O). 


WY, = f(r) At 
Yo, = zf(r) T, 
Wa, = (32?-r’)f(r) E 
Pa =(x—y f(r) E 
Ya, = xyf(r) T> 


The function x transforms according to the irrep T;. Use the above data to 
determine whether dipole matrix elements of the form J = f{ ¢:xd2dt can be 
non-zero for the following pairs of orbitals $;,¢2 in a sulphur hexafluoride 
molecule: (a) ‘Pa, Ps; (b) Yar, Pps (c) Par, Pars (d) ‘Ps, Pass (©) Po, Ps- 
The hydrogen atoms in a methane molecule CH, form a perfect tetrahedron 
with the carbon atom at its centre. The molecule is most conveniently described 
mathematically by placing the hydrogen atoms at the points (1,1, 1), (1,—1,—1), 
(—1,1,-—1) and (—1,—1, 1). The symmetry group to which it belongs, the tetrahe- 
dral group (43m or T,), has classes typified by I, 3, 2., mg and 4,, where the first 
three are as in exercise 29.5, mq is a reflection in the mirror plane x — y = 0 and 
4, is a rotation of 2/2 about the z-axis followed by an inversion in the origin. A 
reflection in a mirror plane can be considered as a rotation of z about an axis 
perpendicular to the plane, followed by an inversion in the origin. 

The character table for the group 43m is very similar to that for the group 
432, and has the form shown in table 29.9. 


Typical element and class size Functions transforming 
4. ma according to irrep 


No 
nN 


x? + y? + 22 


(x? _ y’, 37? fx r) 
(Rx, Ry, Rz) 
(X,¥,2)3 (XY, YZ, 2X) 


3 
8 
1 
1 
—1 
0 
0 


BREN D 


Table 29.9 The character table for group 43m. 


By following the steps given below, determine how many different internal vibra- 
tion frequencies the CH, molecule has. 


(a) Consider a representation based on the twelve coordinates x;, yi, z; for 
i = 1,2,3,4. For those hydrogen atoms that transform into themselves, a 
rotation through an angle @ about an axis parallel to one of the coordinate 
axes gives rise in the natural representation to the diagonal elements 1 for 
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the corresponding coordinate and 2 cos 0 for the two orthogonal coordinates. 
If the rotation is followed by an inversion then these entries are multiplied 
by —1. Atoms not transforming into themselves give a zero diagonal contri- 
bution. Show that the characters of the natural representation are 12, 0, 0, 
0, 2 and hence that its expression in terms of irreps is 


A, ®@E@T, @ 27. 


(b) The irreps of the bodily translational and rotational motions are included in 
this expression and need to be identified and removed. Show that when this 
is done it can be concluded that there are three different internal vibration 
frequencies in the CH, molecule. State their degeneracies and check that 
they are consistent with the expected number of normal coordinates needed 
to describe the internal motions of the molecule. 


Investigate the properties of an alternating group and construct its character 
table as follows. 


(a) The set of even permutations of four objects (a proper subgroup of S,) 
is known as the alternating group A4. List its twelve members using cycle 
notation. 

(b) Assume that all permutations with the same cycle structure belong to the 
same conjugacy class. Show that this leads to a contradiction, and hence 
demonstrates that, even if two permutations have the same cycle structure, 
they do not necessarily belong to the same class. 

(c) By evaluating the products 


Pi = (123)(4) e (12)(34) e (132)(4) and po = (132)(4) e (12)(34) e (123)(4) 


deduce that the three elements of A4 with structure of the form (12)(34) 
belong to the same class. 

(d) By evaluating products of the form (1a)(By) e (123)(4) e(1a)(By), where a, B, y 
are various combinations of 2, 3, 4, show that the class to which (123)(4) 
belongs contains at least four members. Show the same for (124)(3). 

(e) By combining results (b), (c) and (d) deduce that Aq has exactly four classes, 
and determine the dimensions of its irreps. 

(f) Using the orthogonality properties of characters and noting that elements of 
the form (124)(3) have order 3, find the character table for Ay. 


Use the results of exercise 28.23 to find the character table for the dihedral group 
Ds, the symmetry group of a regular pentagon. 

Demonstrate that equation (29.24) does, indeed, generate a set of vectors trans- 
forming according to an irrep 4, by sketching and superposing drawings of an 
equilateral triangle of springs and masses, based on that shown in figure 29.5. 


(b) (c) 
Figure 29.7 The three normal vibration modes of the equilateral array. Mode 


(a) is known as the ‘breathing mode’. Modes (b) and (c) transform according 
to irrep E and have equal vibrational frequencies. 
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(a) Make an initial sketch showing an arbitrary small mass displacement from, 
say, vertex C. Draw the results of operating on this initial sketch with each 
of the symmetry elements of the group 3m (C;,). 

(b) Superimpose the results, weighting them according to the characters of irrep 
A, (table 29.1 in section 29.6) and verify that the resultant is a symmetrical 
arrangement in which all three masses move symmetrically towards (or away 
from) the centroid of the triangle. The mode is illustrated in figure 29.7(a). 

(c) Start again, this time considering a displacement 6 of C parallel to the x-axis. 
Form a similar superposition of sketches weighted according to the characters 
of irrep E (note that the reflections are not needed). The resultant contains 
some bodily displacement of the triangle, since this also transforms according 
to E. Show that the displacement of the centre of mass is X = 6, 7 = 0. 
Subtract this out, and verify that the remainder is of the form shown in 
figure 29.7(c). 

(d) Using an initial displacement parallel to the y-axis, and an analogous proce- 
dure, generate the remaining normal mode, degenerate with that in (c) and 
shown in figure 29.7(b). 


Further investigation of the crystalline compound considered in exercise 29.7 
shows that the octahedron is not quite perfect but is elongated along the (1, 1, 1) 
direction with the sulphur atoms at positions +(a+6, 6,6), +(6,a+6, 6), +(6,6,a+ 
6), where 6 < a. This structure is invariant under the (crystallographic) symmetry 
group 32 with three two-fold axes along directions typified by (1,—1,0). The 
latter axes, which are perpendicular to the (1,1,1) direction, are axes of two- 
fold symmetry for the perfect octahedron. The group 32 is really the three- 
dimensional version of the group 3m and has the same character table as table 29.1 
(section 29.6). Use this to show that, when the distortion of the octahedron is 
included, the doublet found in exercise 29.7 is unsplit but the triplet breaks up 
into a singlet and a doublet. 


29.13 Hints and answers 


There are four classes and hence four one-dimensional irreps, which must have 
entries as follows: 1, 1, 1, 1; 1, 1, -1, -1; 1, —1, 1, -1; 1, —1, —1, 1. The 
characters of D are 2, —2, 0, 0 and so the irreps present are the last two of these. 
There are five classes {1}, {—1}, {+i}, {+j}, {+k}; there are four one-dimensional 
irreps and one two-dimensional irrep. Show that ab = ba. The homomorphism 
is +1 ~ I, ti ~ a, tj — b, +k — ab. V is Abelian and hence has four 
one-dimensional irreps. 

In the class order given above, the characters for Q are as follows: 
Okie iat ieb od Ltt 

Bo avait 6) 929. 0,00 

Note that the fourth and fifth classes each have 6 members. 

The five basis functions of the representation are multiplied by 1, e~*, et, 
e2i@” e+2ié” as a result of the rotation. The character is the sum of these for 
rotations of 0, 27/3, x, 2/2, 7; DSP =E+T>. 

(b) The bodily translation has irrep T, and the rotation has irrep T;. The irreps 
of the internal vibrations are A;, E, Tz, with respective degeneracies 1, 2, 3, 
making six internal coordinates (12 in total, minus three translational, minus 
three rotational). 

There are four classes and hence four irreps, which can only be the identity 
irrep, one other one-dimensional irrep, and two two-dimensional irreps. In the 
class order {I}, {R,R*}, {R?, R?}, {m;} the second one-dimensional irrep must 
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(because of orthogonality) have characters 1, 1, 1, —1. The summation rules 
and orthogonality require the other two character sets to be 2, (—1 + ./5)/2, 
(—1—./5)/2, 0 and 2, (—1— /5)/2, (-1+ ./5)/2, 0. Note that R has order 5 and 
that, e.g, (—1 + ./5)/2 = exp(27i/5) + exp(8zi/5). 

The doublet irrep E (characters 2, —1, 0) appears in both 432 and 32 and so 
is unsplit. The triplet T, (characters 3, 0, 1) splits under 32 into doublet E 
(characters 2, —1, 0) and singlet A, (characters 1, 1, 1). 
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All scientists will know the importance of experiment and observation and, 
equally, be aware that the results of some experiments depend to a degree on 
chance. For example, in an experiment to measure the heights of a random sample 
of people, we would not be in the least surprised if all the heights were found to 
be different; but, if the experiment were repeated often enough, we would expect 
to find some sort of regularity in the results. Statistics, which is the subject of the 
next chapter, is concerned with the analysis of real experimental data of this sort. 
First, however, we discuss probability. To a pure mathematician, probability is an 
entirely theoretical subject based on axioms. Although this axiomatic approach is 
important, and we discuss it briefly, an approach to probability more in keeping 
with its eventual applications in statistics is adopted here. 

We first discuss the terminology required, with particular reference to the 
convenient graphical representation of experimental results as Venn diagrams. 
The concepts of random variables and distributions of random variables are then 
introduced. It is here that the connection with statistics is made; we assert that 
the results of many experiments are random variables and that those results have 
some sort of regularity, which is represented by a distribution. Precise definitions 
of a random variable and a distribution are then given, as are the defining 
equations for some important distributions. We also derive some useful quantities 
associated with these distributions. 


30.1 Venn diagrams 


We call a single performance of an experiment a trial and each possible result 
an outcome. The sample space S of the experiment is then the set of all possible 
outcomes of an individual trial. For example, if we throw a six-sided die then there 
are six possible outcomes that together form the sample space of the experiment. 
At this stage we are not concerned with how likely a particular outcome might 
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Figure 30.1 A Venn diagram. 


be (we will return to the probability of an outcome in due course) but rather 
will concentrate on the classification of possible outcomes. It is clear that some 
sample spaces are finite (e.g. the outcomes of throwing a die) whilst others are 
infinite (e.g. the outcomes of measuring people’s heights). Most often, one is not 
interested in individual outcomes but in whether an outcome belongs to a given 
subset A (say) of the sample space S; these subsets are called events. For example, 
we might be interested in whether a person is taller or shorter than 180 cm, in 
which case we divide the sample space into just two events: namely, that the 
outcome (height measured) is (i) greater than 180 cm or (ii) less than 180 cm. 

A common graphical representation of the outcomes of an experiment is the 
Venn diagram. A Venn diagram usually consists of a rectangle, the interior of 
which represents the sample space, together with one or more closed curves inside 
it. The interior of each closed curve then represents an event. Figure 30.1 shows 
a typical Venn diagram representing a sample space S and two events A and 
B. Every possible outcome is assigned to an appropriate region; in this example 
there are four regions to consider (marked i to iv in figure 30.1): 


outcomes that belong to event A but not to event B; 
outcomes that belong to event B but not to event A; 
outcomes that belong to both event A and event B; 

outcomes that belong to neither event A nor event B. 


>A six-sided die is thrown. Let event A be ‘the number obtained is divisible by 2’ and event 


B be ‘the number obtained is divisible by 3’. Draw a Venn diagram to represent these events. 


It is clear that the outcomes 2, 4, 6 belong to event A and that the outcomes 3, 6 belong 
to event B. Of these, 6 belongs to both A and B. The remaining outcomes, 1, 5, belong to 
neither A nor B. The appropriate Venn diagram is shown in figure 30.2. < 


In the above example, one outcome, 6, is divisible by both 2 and 3 and so 
belongs to both A and B. This outcome is placed in region iii of figure 30.1, which 
is called the intersection of A and B and is denoted by 41 B (see figure 30.3(a)). 
If no events lie in the region of intersection then A and B are said to be mutually 
exclusive or disjoint. In this case, often the Venn diagram is drawn so that the 
closed curves representing the events A and B do not overlap, so as to make 
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Figure 30.2 The Venn diagram for the outcomes of the die-throwing trials 
described in the worked example. 
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Figure 30.3 Venn diagrams: the shaded regions show (a) AM B, the inter- 
section of two events A and B, (b) AUB, the union of events A and B, (c) 
the complement A of an event A, (d) A— B, those outcomes in A that do not 
belong to B. 


graphically explicit the fact that A and B are disjoint. It is not necessary, however, 
to draw the diagram in this way, since we may simply assign zero outcomes to 
the shaded region in figure 30.3(a). An event that contains no outcomes is called 
the empty event and denoted by 0. The event comprising all the elements that 
belong to either A or B, or to both, is called the union of A and B and is denoted 
by AU B (see figure 30.3(b)). In the previous example, AU B = {2, 3, 4, 6}. 
It is sometimes convenient to talk about those outcomes that do not belong to 
a particular event. The set of outcomes that do not belong to A is called the 
complement of A and is denoted by A (see figure 30.3(c)); this can also be written 
as A=S—A. It is clear that AUA=S and ANA=9. 

The above notation can be extended in an obvious way, so that A — B denotes 
the outcomes in A that do not belong to B. It is clear from figure 30.3(d) that 
A-—B can also be written as AM B. Finally, when all the outcomes in event B 
(say) also belong to event A, but A may contain, in addition, outcomes that do 
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Figure 30.4 The general Venn diagram for three events is divided into eight 
regions. 


not belong to B, then B is called a subset of A, a situation that is denoted by 
Bc A; alternatively, one may write A > B, which states that A contains B. In this 
case, the closed curve representing the event B is often drawn lying completely 
within the closed curve representing the event A. 

The operations U and /™ are extended straightforwardly to more than two 
events. If there exist n events A,,A>,...,4,, in some sample space S, then the 
event consisting of all those outcomes that belong to one or more of the A; is the 
union of Aj, Ao2,...,An and is denoted by 


A GAUSS HO Ay (30.1) 


Similarly, the event consisting of all the outcomes that belong to every one of the 
A; is called the intersection of A;,A2,...,A, and is denoted by 


Ay NAQN+++ Ay. (30.2) 
If, for any pair of values i, j with i ¥ j, 


then the events A; and A; are said to be mutually exclusive or disjoint. 

Consider three events A, B and C with a Venn diagram such as is shown in 
figure 30.4. It will be clear that, in general, the diagram will be divided into eight 
regions and they will be of four different types. Three regions correspond to a 
single event; three regions are each the intersection of exactly two events; one 
region is the three-fold intersection of all three events; and finally one region 
corresponds to none of the events. Let us now consider the numbers of different 
regions in a general n-event Venn diagram. 

For one-event Venn diagrams there are two regions, for the two-event case 
there are four regions and, as we have just seen, for the three-event case there are 
eight. In the general n-event case there are 2” regions, as is clear from the fact 
that any particular region R lies either inside or outside the closed curve of any 
particular event. With two choices (inside or outside) for each of n closed curves, 
there are 2” different possible combinations with which to characterise R. Once n 
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gets beyond three it becomes impossible to draw a simple two-dimensional Venn 
diagram, but this does not change the results. 

The 2” regions will break down into n+1 types, with the numbers of each type 
as follows’ 


no events, "Cy = 
one event but no intersections, "C; = 
two-fold intersections, "C) = 


| 
NF Ss Re 


n(n — 1); 
three-fold intersections, "C3 = ann — 1)(n—2); 


an n-fold intersection, "C, = 1. 


That this makes a total of 2" can be checked by considering the binomial 
expansion 


2" =(14+1)"=1+n4 pn(n—1)4+---4+1. 


Using Venn diagrams, it is straightforward to show that the operations N and 
U obey the following algebraic laws: 


commutativity, ANB=BNA, AUB=BUA; 
associativity, (ANB)NC=AN(BNC), (AUB)UC=AU(BUQC); 
distributivity, AN(BUC) =(ANB)U(ANC), 
AU(BNC) =(AUB)N(AUC); 
idempotency, ANA=A, AUA=A. 


> Show that (i) AU(ANB) = AN (AUB) =A, (ii) (A— B)U(ANB) =A. 


(i) Using the distributivity and idempotency laws above, we see that 
AU(ANB) =(AUA)N(AUB)=AN(AUB). 


By sketching a Venn diagram it is immediately clear that both expressions are equal to 
A. Nevertheless, we here proceed in a more formal manner in order to deduce this result 
algebraically. Let us begin by writing 


X =AU(ANB) =AN(AUB), (30.4) 


from which we want to deduce a simpler expression for the event X. Using the first equality 
in (30.4) and the algebraic laws for N and U, we may write 
ANX =AN[AU(ANB)] 
= (AN A)U[AN (ANB) 
=AU(ANB)=X. 


S The symbols "Cj, for i = 0,1,2,...,n, are a convenient notation for combinations; they and their 
properties are discussed in chapter 1. 
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Since AM X = X we must have X c A. Now, using the second equality in (30.4) in a 
similar way, we find 
AUX =AU[AN(AUB)] 
=(AUA)N [AU (AUB)] 
=AN(AUB)=X, 
from which we deduce that A c X. Thus, since X c A and A c X, we must conclude that 
X =A. 

(ii) Since we do not know how to deal with compound expressions containing a minus 
sign, we begin by writing A— B = AMB as mentioned above. Then, using the distributivity 
law, we obtain 

(A— B)U(ANB) =(AN B)U(ANB) 
=AN(BUB) 
=ANS=A. 


In fact, this result, like the first one, can be proved trivially by drawing a Venn diagram. < 


Further useful results may be derived from Venn diagrams. In particular, it is 
simple to show that the following rules hold: 


(i) if Ac B then ADB; 
(ii) AUB=ANB; 
(iii) ANB = AUB. 
Statements (il) and (iii) are known jointly as de Morgan’s laws and are sometimes 
useful in simplifying logical expressions. 


> There exist two events A and B such that 
(XU A) U(X UA) = B. 


Find an expression for the event X in terms of A and B. 


We begin by taking the complement of both sides of the above expression: applying de 
Morgan’s laws we obtain 
B=(XUA)N(X UA). 
We may then use the algebraic laws obeyed by M and U to yield 
B=XU(ANA)=XU O=X. 
Thus, we find that X = B. < 


30.2 Probability 


In the previous section we discussed Venn diagrams, which are graphical repre- 
sentations of the possible outcomes of experiments. We did not, however, give 
any indication of how likely each outcome or event might be when any particular 
experiment is performed. Most experiments show some regularity. By this we 
mean that the relative frequency of an event is approximately the same on each 
occasion that a set of trials is performed. For example, if we throw a die N 
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times then we expect that a six will occur approximately N/6 times (assuming, 
of course, that the die is not biased). The regularity of outcomes allows us to 
define the probability, Pr(A), as the expected relative frequency of event A in a 
large number of trials. More quantitatively, if an experiment has a total of ns 
outcomes in the sample space S, and ny of these outcomes correspond to the 
event A, then the probability that event A will occur is 


Pr(A) = - (30.5) 
30.2.1 Axioms and theorems 
From (30.5) we may deduce the following properties of the probability Pr(A). 
(i) For any event A in a sample space S, 
0 < Pr(A) <1. (30.6) 


If Pr(A) = 1 then A is a certainty; if Pr(A) = 0 then A is an impossibility. 
(ii) For the entire sample space S we have 


Pj St (30.7) 
ns 
which simply states that we are certain to obtain one of the possible 
outcomes. 
(iii) If A and B are two events in S then, from the Venn diagrams in figure 30.3, 
we see that 


N4guB = N4 + Ng — Nan, (30.8) 


the final subtraction arising because the outcomes in the intersection of 
A and B are counted twice when the outcomes of A are added to those 
of B. Dividing both sides of (30.8) by ns, we obtain the addition rule for 
probabilities 


Pr(A U B) = Pr(A) + Pr(B) — P(A B). (30.9) 


However, if A and B are mutually exclusive events (AN B = 0) then 
Pr(A 1 B) = 0 and we obtain the special case 


Pr(A U B) = Pr(A) + Pr(B). (30.10) 


(iv) If A is the complement of A then A and A are mutually exclusive events. 
Thus, from (30.7) and (30.10) we have 


1 = Pr(S) = Pr(A U A) = Pr(A) + Pr(A), 
from which we obtain the complement law 


Pr(A) = 1 — Pr(A). (30.11) 
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This is particularly useful for problems in which evaluating the probability 
of the complement is easier than evaluating the probability of the event 
itself. 


> Calculate the probability of drawing an ace or a spade from a pack of cards. 


Let A be the event that an ace is drawn and B the event that a spade is drawn. It 
immediately follows that Pr(A) = 4 = 4 and Pr(B) = 3 = i The intersection of A and 


1 

52 BB 

B consists of only the ace of spades and so Pr(AN B) = > Thus, from (30.9) 
Pr(4UB)=4+4-f=4. 


In this case it is just as simple to recognise that there are 16 cards in the pack that satisfy 
16 


the required condition (13 spades plus three other aces) and so the probability is =. < 


The above theorems can easily be extended to a greater number of events. For 


example, if A;,A,...,A, are mutually exclusive events then (30.10) becomes 
Pr(A; U Ap U-+- UU An) = Pr(A1) + Pr(Az) +--+ + Pr(An). (30.12) 
Furthermore, if Aj, Ao,...,An (whether mutually exclusive or not) exhaust S, Le. 


are such that A; UA) U---UA, =S, then 


Pr(Ay U A) U-+»U Ay) = Pr(S) = 1. (30.13) 


&A biased six-sided die has probabilities 5D, DP, P, P, p, 2p of showing 1, 2, 3, 4, 5, 6 
respectively. Calculate p. 


Given that the individual events are mutually exclusive, (30.12) can be applied to give 
Pr(1 U2U3U4U5U6) = sp +pt+pt+p+pt2p= Bp. 


The union of all possible outcomes on the LHS of this equation is clearly the sample 
space, S, and so 


Pr($) = Bp. 
Now using (30.7), 


Bp Pr(S) = 1 p=i.< 


When the possible outcomes of a trial correspond to more than two events, 
and those events are not mutually exclusive, the calculation of the probability of 
the union of a number of events is more complicated, and the generalisation of 
the addition law (30.9) requires further work. Let us begin by considering the 
union of three events A;, 42 and A3, which need not be mutually exclusive. We 
first define the event B = Az U A; and, using the addition law (30.9), we obtain 


Pr(Ay U Ap U A3) = Pr(A, U B) = Pr(41) + Pr(B) — Pr(A; 9 B). 
(30.14) 
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However, we may write Pr(A; 9 B) as 


Pr(A; 9 B) = Pr[4, N (A2 U A3)] 
= Pr[(A1 0 42) U(A1 9 A3)] 
= Pr(A, M Az) + Pr(A, 1. A3) — Pr(A, MN A2 1 A3). 


Substituting this expression, and that for Pr(B) obtained from (30.9), into (30.14) 
we obtain the probability addition law for three general events, 


Pr(A; U Ay U A3) = Pr(Az) + Pr(A2) + Pr(A3) — Pr(A2 9 A3) — Pr(A1 N A3) 
—Pr(Ay A Ap) + Pr(Ay 1. Ap 3). (30.15) 


> Calculate the probability of drawing from a pack of cards one that is an ace or is a spade 
or shows an even number (2, 4, 6, 8, 10). 


If, as previously, A is the event that an ace is drawn, Pr(A) = 4. Similarly the event B, 
that a spade is drawn, has Pr(B) = 2. The further possibility C, that the card is even (but 
not a picture card) has Pr(C) = 2. The two-fold intersections have probabilities 


Pr(AN B) = = Pr(ANC)=0, PrBOC)= = 


There is no three-fold intersection as events A and C are mutually exclusive. Hence 


31 


P(AUBUC) =o [4+ 13420-14045) +O] = 3. 


52 


The reader should identify the 31 cards involved. < 


When the probabilities are combined to calculate the probability for the union 
of the n general events, the result, which may be proved by induction upon n (see 
the answer to exercise 30.4), is 


Pr(Ay U A? U ++ U An) = 9» Pr(Ai) — 9° Pr(4i 0 Ay) + > Pr(Ai Aj Ax) 
i ij i,j,k 
= (1 Pray As «0 A}), (30.16) 


Each summation runs over all possible sets of subscripts, except those in which 
any two subscripts in a set are the same. The number of terms in the summation 
of probabilities of m-fold intersections of the n events is given by "C,,, (as discussed 
in section 30.1). Equation (30.9) is a special case of (30.16) in which n = 2 and 
only the first two terms on the RHS survive. We now illustrate this result with a 
worked example that has n = 4 and includes a four-fold intersection. 
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> Find the probability of drawing from a pack a card that has at least one of the following 
properties: 

A, it is an ace; 

B, it is a spade; 

C, it is a black honour card (ace, king, queen, jack or 10); 

D, it is a black ace. 


Measuring all probabilities in units of > the single-event probabilities are 
Pr(A) = 4, Pr(B) = 13, Pr(C) = 10, Pr(D) = 2. 
The two-fold intersection probabilities, measured in the same units, are 
Pr(A/N B) = 1, Pr(ANC) = 2, Pr(AN D) = 2, 
Pr(B NC) =5, Pr(BO D) = 1, Pr(C ND) =2. 
The three-fold intersections have probabilities 
Pr(ANBNC)=1, PrANBND)=1, PrANCND)=2, Pr(BNCND)=1. 


Finally, the four-fold intersection, requiring all four conditions to hold, is satisfied only by 
the ace of spades, and hence (again in units of 3) 


Pr(ANBNCND)=1. 
Substituting in (30.16) gives 
1 20 

P=Z[4+13+ 1042-1 424245414241 4142+ )-M=5. < 

We conclude this section on basic theorems by deriving a useful general 
expression for the probability Pr(A M B) that two events A and B both occur in 
the case where A (say) is the union of a set of n mutually exclusive events A;. In 
this case 

ANB =(A,NB)U---U(A, 1B), 


where the events A; 1 B are also mutually exclusive. Thus, from the addition law 
(30.12) for mutually exclusive events, we find 


Pr(AMB) =) Pr(4j 7B). (30.17) 
i 
Moreover, in the special case where the events A; exhaust the sample space S, we 
have AN B = SB =B, and we obtain the total probability law 


Pr(B) = 5) Pr(A; B). (30.18) 


30.2.2 Conditional probability 


So far we have defined only probabilities of the form ‘what is the probability that 
event A happens?’. In this section we turn to conditional probability, the probability 
that a particular event occurs given the occurrence of another, possibly related, 
event. For example, we may wish to know the probability of event B, drawing an 
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ace from a pack of cards from which one has already been removed, given that 
event A, the card already removed was itself an ace, has occurred. 

We denote this probability by Pr(B|A) and may obtain a formula for it by 
considering the total probability Pr(4 9 B) = Pr(B NM A) that both A and B will 
occur. This may be written in two ways, Le. 


Pr(A NB) = Pr(A) Pr(BIA) 


= Pr(B) Pr(A|B). 
From this we obtain 
Pr(4|B) = ree (30.19) 
and 
Pr(B|A) = ae (30.20) 


In terms of Venn diagrams, we may think of Pr(B|A) as the probability of B in 
the reduced sample space defined by A. Thus, if two events A and B are mutually 
exclusive then 


Pr(A|B) = 0 = Pr(BIA). (30.21) 


When an experiment consists of drawing objects at random from a given set 
of objects, it is termed sampling a population. We need to distinguish between 
two different ways in which such a sampling experiment may be performed. After 
an object has been drawn at random from the set it may either be put aside 
or returned to the set before the next object is randomly drawn. The former is 
termed ‘sampling without replacement’, the latter ‘sampling with replacement’. 


> Find the probability of drawing two aces at random from a pack of cards (i) when the 
first card drawn is replaced at random into the pack before the second card is drawn, and 


(ii) when the first card is put aside after being drawn. 


Let A be the event that the first card is an ace, and B the event that the second card is an 
ace. Now 


Pr(4 B) = Pr( els) 


and for both (i) and (ii) we know that Pr(A) = 4 = 4 
(i) If the first card is replaced in the pack bein. the next is drawn then Pr(B|A) = 


Pr(B) = s = ty since A and B are independent events. We then have 
Pr(A M B) = Pr(A) Pr(B) = : x ees 
7 ~ 137 13 169" 


(ii) If the first card is put aside and the second then drawn, A and B are not independent 


and Pr(B|A) = 3, with the result that 


Pr(AN B) = Pr(A) Pr(B|A) = a x = a 
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Two events A and B are statistically independent if Pr(A|B) = Pr(A) (or equiva- 
lently if Pr(B|A) = Pr(B)). In words, the probability of A given B is then the same 
as the probability of A regardless of whether B occurs. For example, if we throw 
a coin and a die at the same time, we would normally expect that the probability 
of throwing a six was independent of whether a head was thrown. If A and B are 
statistically independent then it follows that 


Pr(A NB) = Pr(A) Pr(B). (30.22) 


In fact, on the basis of intuition and experience, (30.22) may be regarded as the 
definition of the statistical independence of two events. 

The idea of statistical independence is easily extended to an arbitrary number 
of events 41, A2,..., An. The events are said to be (mutually) independent if 


Pr(4pn Aj) = Pr(Aj) Pr(A,), 
Pr(A; A; 1 Ay) = Pr(A;) Pr(A,) Pr(Ay), 


Pr(4y N42 A+++ An) = Pr(Aq) Pr(A2) «+» Pr(A,), 


for all combinations of indices i, j and k for which no two indices are the same. 
Even if all n events are not mutually independent, any two events for which 
Pr(A; M.A;) = Pr(Aj) Pr(A;) are said to be pairwise independent. 

We now derive two results that often prove useful when working with condi- 
tional probabilities. Let us suppose that an event A is the union of n mutually 
exclusive events A;. If B is some other event then from (30.17) we have 


Pr(AB) =) Pr(4; 9B). 


Dividing both sides of this equation by Pr(B), and using (30.19), we obtain 
Pr(4|B) = ) > Pr(Ai|B), (30.23) 


which is the addition law for conditional probabilities. 

Furthermore, if the set of mutually exclusive events A; exhausts the sample 
space S then, from the total probability law (30.18), the probability Pr(B) of some 
event B in S can be written as 


Pr(B) = > Pr(A;) Pr(B|A)). (30.24) 


>A collection of traffic islands connected by a system of one-way roads is shown in fig- 
ure 30.5. At any given island a car driver chooses a direction at random from those available. 
What is the probability that a driver starting at O will arrive at B? 


In order to leave O the driver must pass through one of A;, 42, A3 or Ag, which thus 
form a complete set of mutually exclusive events. Since at each island (including O) the 
driver chooses a direction at random from those available, we have that Pr(A;) = ; for 
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Figure 30.5 A collection of traffic islands connected by one-way roads. 


i= 1,2,3,4. From figure 30.5, we see also that 

Pr(BlA\) = 3, Pr(Bl42)= 4, Pr(BlA3)=0, Pr(BlAg) = § = 5. 
Thus, using the total probability law (30.24), we find that the probability of arriving at B 
is given by 


Pr(B) = de Pala )Pr(Bl4) =4(£+44+0+4+1)=1.< 


Finally, we note that the concept of conditional probability may be straightfor- 
wardly extended to several compound events. For example, in the case of three 
events A, B, C, we may write Pr(AM BMC) in several ways, e.g. 


Pr(AN BNC) = Pr(C) Pr(An BIC) 
= Pr(BN C)Pr(A|BNC) 
= Pr(C) Pr(B|C) Pr(A|B C). 


> Suppose {A;\ is a set of mutually exclusive events that exhausts the sample space S. If B 
and C are two other events in S, show that 


Pr(B|C) = ) > Pr(Ai|C) Pr(B A; C). 


Using (30.19) and (30.17), we may write 
Pr(C) Pr(B|C) = Pr(B OC) = iy Pr(A; A BOC). (30.25) 
Each term in the sum on the RHS can be expanded as an appropriate product of 
conditional probabilities, 
Pr(Aj N BMC) = Pr(C) Pr(A;|C) Pr(B|A;N C). 


Substituting this form into (30.25) and dividing through by Pr(C) gives the required 
result. < 
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30.2.3 Bayes’ theorem 


In the previous section we saw that the probability that both an event A and a 
related event B will occur can be written either as Pr(A) Pr(B|A) or Pr(B) Pr(A|B). 
Hence 


Pr(A) Pr(B|A) = Pr(B) Pr(A|B), 
from which we obtain Bayes’ theorem, 


Pr(A) 


Pr(A|B) = Pr(B) 


Pr(BIA). (30.26) 


This theorem clearly shows that Pr(B|A) # Pr(A|B), unless Pr(A) = Pr(B). It is 
sometimes useful to rewrite Pr(B), if it is not known directly, as 


Pr(B) = Pr(A) Pr(B|A) + Pr(A) Pr(B|A) 
so that Bayes’ theorem becomes 


Pr(A|B) = Ee Pl) =, (30.27) 
Pr(A) Pr(B|A) + Pr(A) Pr(B|A) 


> Suppose that the blood test for some disease is reliable in the following sense: for people 
who are infected with the disease the test produces a positive result in 99.99% of cases; for 
people not infected a positive test result is obtained in only 0.02% of cases. Furthermore, 
assume that in the general population one person in 10000 people is infected. A person is 
selected at random and found to test positive for the disease. What is the probability that 
the individual is actually infected? 


Let A be the event that the individual is infected and B be the event that the individual 
tests positive for the disease. Using Bayes’ theorem the probability that a person who tests 
positive is actually infected is 


pom Pr(A) Pr(B|A) 
Pr(A) Pr(B|A) + Pr(A) Pr(B|A) 


Now Pr(A) = 1/10000 = 1 — Pr(A), and we are told that Pr(B\A) = 9999/10000 and 


Pr(B|A) = 2/10000. Thus we obtain 


1/10000 x 9999/10000 1 


a a (1/10000 x 9999/10000) + (9999/10000 x 2/10000) 3” 
Thus, there is only a one in three chance that a person chosen at random, who tests 
positive for the disease, is actually infected. 

At a first glance, this answer may seem a little surprising, but the reason for the counter- 
intuitive result is that the probability that a randomly selected person is not infected is 
9999 /10000, which is very high. Thus, the 0.02% chance of a positive test for an uninfected 
person becomes significant. < 
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We note that (30.27) may be written in a more general form if S is not simply 
divided into A and A but, rather, into any set of mutually exclusive events A; that 
exhaust S. Using the total probability law (30.24), we may then write 


Pr(B) = > Pr(Ai) Pr(BIAi), 


so that Bayes’ theorem takes the form 
Pr(A) Pr(B|A) 
>3; Pr(Ai) Pr(BlAi)’ 
where the event A need not coincide with any of the Aj. 
As a final point, we comment that sometimes we are concerned only with the 
relative probabilities of two events A and C (say), given the occurrence of some 
other event B. From (30.26) we then obtain a different form of Bayes’ theorem, 
Pr(A|B) — Pr(A) Pr(BIA) 
Pr(C|B)  Pr(C) Pr(BIC)’ 


which does not contain Pr(B) at all. 


Pr(A|B) = (30.28) 


(30.29) 


30.3. Permutations and combinations 
In equation (30.5) we defined the probability of an event A in a sample space S as 
Pr(4) = 4, 
ng 
where ny is the number of outcomes belonging to event A and ng is the total 


number of possible outcomes. It is therefore necessary to be able to count the 
number of possible outcomes in various common situations. 


30.3.1 Permutations 


Let us first consider a set of n objects that are all different. We may ask in 
how many ways these n objects may be arranged, ic. how many permutations of 
these objects exist. This is straightforward to deduce, as follows: the object in the 
first position may be chosen in n different ways, that in the second position in 
n—1 ways, and so on until the final object is positioned. The number of possible 
arrangements is therefore 


n(n — 1)(n—2)-+-(1) =n! (30.30) 


Generalising (30.30) slightly, let us suppose we choose only k (< n) objects 
from n. The number of possible permutations of these k objects selected from n 
is given by 


n! 


n(n — 1)(n— 2)---(n—k +1) = —— ='P,. (30.31) 
(nk)! 
k factors 
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In calculating the number of permutations of the various objects we have so 
far assumed that the objects are sampled without replacement — i.e. once an object 
has been drawn from the set it is put aside. As mentioned previously, however, 
we may instead replace each object before the next is chosen. The number of 
permutations of k objects from n with replacement may be calculated very easily 
since the first object can be chosen in n different ways, as can the second, the 
third, etc. Therefore the number of permutations is simply n‘. This may also be 
viewed as the number of permutations of k objects from n where repetitions are 
allowed, i.e. each object may be used as often as one likes. 


> Find the probability that in a group of k people at least two have the same birthday 
(ignoring 29 February ). 


It is simplest to begin by calculating the probability that no two people share a birthday, 
as follows. Firstly, we imagine each of the k people in turn pointing to their birthday on 
a year planner. Thus, we are sampling the 365 days of the year ‘with replacement’ and so 
the total number of possible outcomes is (365)*. Now (for the moment) we assume that no 
two people share a birthday and imagine the process being repeated, except that as each 
person points out their birthday it is crossed off the planner. In this case, we are sampling 
the days of the year ‘without replacement’, and so the possible number of outcomes for 
which all the birthdays are different is 


365! 
365, — 
*~ B65—k)! 
Hence the probability that all the birthdays are different is 
365! 
P 


~ B65—K)! 365" 
Now using the complement rule (30.11), the probability q that two or more people have 
the same birthday is simply 
365! 
=1-p=1-———"—.. 
7 x (365 —k)! 365° 


This expression may be conveniently evaluated using Stirling’s approximation for n! when 


n is large, namely 
n!~ ./2nn (=) ; 
e 


to give 


365 365—k+0.5 
x 1l—e*( ——_ . 
4 7 ( 365 — i) 


It is interesting to note that if k = 23 the probability is a little greater than a half that 
at least two people have the same birthday, and if k = 50 the probability rises to 0.970. 
This can prove a good bet at a party of non-mathematicians! < 


So far we have assumed that all n objects are different (or distinguishable). Let 
us now consider n objects of which n; are identical and of type 1, nz are identical 
and of type 2, ...,m,, are identical and of type m (clearly n =n; +n) +--+ +1). 
From (30.30) the number of permutations of these n objects is again n!. However, 
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the number of distinguishable permutations is only 


n! 
Aleiceaa (30.32) 
since the ith group of identical objects can be rearranged in n;! ways without 
changing the distinguishable permutation. 


In total there are 22 balls, the 15 reds being indistinguishable. Thus from (30.32) the 
number of distinguishable permutations is 


22! 22! 


859 541 760. < 


G)G)G)aDd Dads) 15! 


30.3.2 Combinations 


We now consider the number of combinations of various objects when their order 
is immaterial. Assuming all the objects to be distinguishable, from (30.31) we see 
that the number of permutations of k objects chosen from n is "P, =n!/(n—k)!. 
Now, since we are no longer concerned with the order of the chosen objects, which 
can be internally arranged in k! different ways, the number of combinations of k 
objects from n is 


n! 


n = n 
(n—k)ikl c= ( ) for0<k<n, (30.33) 


k 
where, as noted in chapter 1, "C; is called the binomial coefficient since it also 
appears in the binomial expansion for positive integer n, namely 
n 
(a+ by" = So"Catbr*. (30.34) 
k=0 


>A hand of 13 playing cards is dealt from a well-shuffled pack of 52. What is the probability 
that the hand contains two aces? 


Since the order of the cards in the hand is immaterial, the total number of distinct hands 
is simply equal to the number of combinations of 13 objects drawn from 52, ie. *Cy3. 
However, the number of hands containing two aces is equal to the number of ways, 4C», 
in which the two aces can be drawn from the four available, multiplied by the number of 
ways, 48C1,, in which the remaining 11 cards in the hand can be drawn from the 48 cards 
that are not aces. Thus the required probability is given by 


40, 8C;, 4! 48! 13139! 

BC) 212! 11137! 52! 
(3)(4) (12)(13)(38)(39) 
2 (49)(50)(51)(52) 


0.213 < 
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Another useful result that may be derived using the binomial coefficients is the 
number of ways in which n distinguishable objects can be divided into m piles, 
with n; objects in the ith pile, i = 1,2,...,m (the ordering of objects within each 
pile being unimportant). This may be straightforwardly calculated as follows. We 
may choose the n; objects in the first pile from the original n objects in "C,,, ways. 
The nz objects in the second pile can then be chosen from the n — n, remaining 
objects in """'C,,, ways, etc. We may continue in this fashion until we reach the 
(m — 1)th pile, which may be formed in """'~"~""2C,,_, ways. The remaining 
objects then form the mth pile and so can only be ‘chosen’ in one way. Thus the 
total number of ways of dividing the original n objects into m piles is given by 
the product 


_n n—n N—N—++-—Nm—2 
N= Ch, Ch eae a Cams 


= n! (n—n)! (n— ny — my —--+—Ny_2)! 
ny '(n— ny)! n(n — ny — ny)! Nn—1'\(n — Ny — Ny — +++ — Mp2 — Nn-1)! 
_ n! (n—n)! _ (an =n +++ —Mn-2)! 
ny '(n— ny)! n(n — ny — np)! Nn—1 Nm ! 
= ae a (30.35) 
ny !nz!+-+-Np! 


These numbers are called multinomial coefficients since (30.35) is the coefficient of 
ny nz 
XY XX 


integer n 


mn 


mm in the multinomial expansion of (x; +x2+--:+X,)", Le. for positive 


n! 

" > NTs ny nz wn, 
(x1 +X2 +--+ +Xm)" = ) St 0 Xn" 
NY NY, sm My Ngs ++ Nm: 
nytng+++++nm=n 


For the case m = 2, n, =k, no = n—k, (30.35) reduces to the binomial coefficient 
"C,. Furthermore, we note that the multinomial coefficient (30.35) is identical to 
the expression (30.32) for the number of distinguishable permutations of n objects, 
n; of which are identical and of type i (for i = 1,2,...,m and ny +n+---+n, =n). 
A few moments’ thought should convince the reader that the two expressions 
(30.35) and (30.32) must be identical. 


In the card game of bridge, each of four players is dealt 13 cards from a full pack of 52. 
What is the probability that each player is dealt an ace? 


From (30.35), the total number of distinct bridge dealings is 52!/(13!13!13!13!). However, 
the number of ways in which the four aces can be distributed with one in each hand is 
4!/(1!1!1!1!) = 4!; the remaining 48 cards can then be dealt out in 48!/(12!12!12!12!) 
ways. Thus the probability that each player receives an ace is 

48! (13!)* | 24(13)4 


“T2)* 52! (49)(50)(51)(52) 


= 0.105. < 


As in the case of permutations we might ask how many combinations of k 
objects can be chosen from n with replacement (repetition). To calculate this, we 
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may imagine the n (distinguishable) objects set out on a table. Each combination 
of k objects can then be made by pointing to k of the n objects in turn (with 
repetitions allowed). These k equivalent selections distributed amongst n different 
but re-choosable objects are strictly analogous to the placing of k indistinguishable 
‘balls’ in n different boxes with no restriction on the number of balls in each box. 
A particular selection in the case k = 7, n = 5 may be symbolised as 


Xxx|  [x|Xx|x. 


This denotes three balls in the first box, none in the second, one in the third, two 
in the fourth and one in the fifth. We therefore need only consider the number of 
(distinguishable) ways in which k crosses and n—1 vertical lines can be arranged, 
ie. the number of permutations of k + n—1 objects of which k are identical 
crosses and n— 1 are identical lines. This is given by (30.33) as 


(k+n—1)! 


“kin—D! = wkd Ck. (30.36) 


We note that this expression also occurs in the binomial expansion for negative 
integer powers. If n is a positive integer, it is straightforward to show that (see 
chapter 1) 


(ee) 


(a+ by" = Soepk MRT Cy gtk pk 
k=0 


where a is taken to be larger than b in magnitude. 


>A system contains a number N of (non-interacting) particles, each of which can be in 
any of the quantum states of the system. The structure of the set of quantum states is such 
that there exist R energy levels with corresponding energies E; and degeneracies g; (i.e. the 
ith energy level contains g; quantum states). Find the numbers of distinct ways in which 
the particles can be distributed among the quantum states of the system such that the ith 
energy level contains n; particles, for i = 1,2,...,R, in the cases where the particles are 


(i) distinguishable with no restriction on the number in each state; 
(ii) indistinguishable with no restriction on the number in each state; 
(iii) indistinguishable with a maximum of one particle in each state; 
(iv) distinguishable with a maximum of one particle in each state. 


It is easiest to solve this problem in two stages. Let us first consider distributing the N 
particles among the R energy levels, without regard for the individual degenerate quantum 
states that comprise each level. If the particles are distinguishable then the number of 
distinct arrangements with n; particles in the ith level, i = 1,2,...,R, is given by (30.35) as 


N! 
ny lno!-+-npl 


If, however, the particles are indistinguishable then clearly there exists only one distinct 
arrangement having n; particles in the ith level, i = 1,2,...,R . If we suppose that there 
exist w; ways in which the n; particles in the ith energy level can be distributed among 
the g; degenerate states, then it follows that the number of distinct ways in which the N 
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particles can be distributed among all R quantum states of the system, with n; particles in 
the ith level, is given by 


R 
N! 
— II w; for distinguishable particles, 
we 1 ny Ing! + R! 1 
Us =F oR - 
Il Wi for indistinguishable particles. (30.37) 


It therefore remains only for us to find the appropriate expression for w; in each of the 
cases (i)-(iv) above. 

Case (i). If there is no restriction on the number of particles in each quantum state, 
then in the ith energy level each particle can reside in any of the g; degenerate quantum 
states. Thus, if the particles are distinguishable then the number of distinct arrangements 
is simply w; = g;'. Thus, from (30.37), 


ny!ny!---np! 


Such a system of particles (for example atoms or molecules in a classical gas) is said to 
obey Maxwell-Boltzmann statistics. 

Case (ii). If the particles are indistinguishable and there is no restriction on the number 
in each state then, from (30.36), the number of distinct arrangements of the n; particles 
among the g; states in the ith energy level is 


—_ (i t+gi—1)! 
nl(gi— A)! 


Substituting this expression in (30.37), we obtain 


(n, + gi — 1)! 
f 
Win} = et n;! “nile — DL 


Such a system of particles (for example a gas of photons) is said to obey Bose-Einstein 
statistics. 

Case (iii). If a maximum of one particle can reside in each of the g; degenerate quantum 
states in the ith energy level then the number of particles in each state is either 0 or 1. 
Since the particles are indistinguishable, w; is equal to the number of distinct arrangements 
in which n; states are occupied and g; — n; states are unoccupied; this is given by 


Thus, from (30.37), we have 


Such a system is said to obey Fermi—Dirac statistics, and an example is provided by an 
electron gas. 

Case (iv). Again, the number of particles in each state is either 0 or 1. If the particles 
are distinguishable, however, each arrangement identified in case (iii) can be reordered in 
n;! different ways, so that 


gi! 


wi= iP, re 
(gi — ni)! 
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Substituting this expression into (30.37) oe 


WON eee ea —n) 


i=1 


Such a system of particles has the names of no famous scientists attached to it, since it 
appears that it never occurs in nature. < 


30.4 Random variables and distributions 


Suppose an experiment has an outcome sample space S. A real variable X that 
is defined for all possible outcomes in S (so that a real number — not necessarily 
unique — is assigned to each possible outcome) is called a random variable (RV). 
The outcome of the experiment may already be a real number and hence a random 
variable, e.g. the number of heads obtained in 10 throws of a coin, or the sum of 
the values if two dice are thrown. However, more arbitrary assignments are possi- 
ble, e.g. the assignment of a ‘quality’ rating to each successive item produced by a 
manufacturing process. Furthermore, assuming that a probability can be assigned 
to all possible outcomes in a sample space S, it is possible to assign a probability 
distribution to any random variable. Random variables may be divided into two 
classes, discrete and continuous, and we now examine each of these in turn. 


30.4.1 Discrete random variables 


A random variable X that takes only discrete values x1, x2, ...,Xn, with proba- 
bilities py, po, ...,Pn, is called a discrete random variable. The number of values 
n for which X has a non-zero probability is finite or at most countably infinite. 
As mentioned above, an example of a discrete random variable is the number of 
heads obtained in 10 throws of a coin. If X is a discrete random variable, we can 
define a probability function (PF) f(x) that assigns probabilities to all the distinct 
values that X can take, such that 


po ifx=x, 


f(x) = Pr(X = x) = (30.38) 


0 otherwise. 


A typical PF (see figure 30.6) thus consists of spikes, at valid values of X, whose 
height at x corresponds to the probability that X = x. Since the probabilities 
must sum to unity, we require 


>> fm) =1. (30.39) 
i=1 


We may also define the cumulative probability function (CPF) of X, F(x), whose 
value gives the probability that X < x, so that 


F(x) =Pr(X <x) => f(x (30.40) 


XjSX 
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f(x) F(x) 
2p : 
pL 
si 
” i 
3 45 6° 123 4 5 6 


Figure 30.6 (a) A typical probability function for a discrete distribution, that 
for the biased die discussed earlier. Since the probabilities must sum to unity 
we require p = 2/13. (b) The cumulative probability function for the same 
discrete distribution. (Note that a different scale has been used for (b).) 


Hence F(x) is a step function that has upward jumps of p; at x = x;, i = 


1,2,...,n, and is constant between possible values of X. We may also calculate 
the probability that X lies between two limits, /, and Jy (J; < l)); this is given by 
Pry <X <hjy= > f (xi) = F(b) — F(h), (30.41) 

<xi<h 


Le. it is the sum of all the probabilities for which x; lies within the relevant interval. 


>A bag contains seven red balls and three white balls. Three balls are drawn at random 
and not replaced. Find the probability function for the number of red balls drawn. 


Let X be the number of red balls drawn. Then 


32s 1 
PAX =0) =f) == xox o=— 
PAX =1)= f= a xoxex3—Z, 
PX =2)=fQ)= a xoxo x3=F, 
7 625 7 
PAX =3)=fQ)= xox r=57 
It should be noted that >}, f(i) = 1, as expected. < 


30.4.2. Continuous random variables 


A random variable X is said to have a continuous distribution if X is defined for a 
continuous range of values between given limits (often —oo to 00). An example of 
a continuous random variable is the height of a person drawn from a population, 
which can take any value (within limits!). We can define the probability density 
function (PDF) f(x) of a continuous random variable X such that 


Pr(x < X < x+dx) = f(x) dx, 


1140 


30.4 RANDOM VARIABLES AND DISTRIBUTIONS 


F(x) 


ea 


I a b lL 


Figure 30.7 The probability density function for a continuous random vari- 
able X that can take values only between the limits /; and . The shaded area 
under the curve gives Pr(a < X < b), whereas the total area under the curve, 
between the limits /; and J), is equal to unity. 


ie. f(x) dx is the probability that X lies in the interval x < X < x+dx. Clearly 
f(x) must be a real function that is everywhere > 0. If X can take only values 
between the limits /; and Jy then, in order for the sum of the probabilities of all 
possible outcomes to be equal to unity, we require 


Often X can take any value between —co and o and so 


“ f(x) dx = 1. 


The probability that X lies in the interval a < X < b is then given by 
b 
Pr(a< X <b) = i) f(x) dx, (30.42) 
a 
ie. Pr(a < X < b) is equal to the area under the curve of f(x) between these 
limits (see figure 30.7). 


We may also define the cumulative probability function F(x) for a continuous 
random variable by 


F(x) =Pr(X <x) = /  f(u) du, (30.43) 
I 
where u is a (dummy) integration variable. We can then write 
Pr(a < X < b) = F(b) — F(a). 
From (30.43) it is clear that f(x) = dF(x)/dx. 
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>A random variable X has a PDF f(x) given by Ae~ in the interval 0 < x < ~ and zero 
elsewhere. Find the value of the constant A and hence calculate the probability that X lies 
in the interval 1 < X <2. 


We require the integral of f(x) between 0 and o to equal unity. Evaluating this integral, 
we find 


/ As gee [—Ae*] > =A, 
0 
and hence A = 1. From (30.42), we then obtain 


2 2 
Pr(l < X <2)= ‘ f(x) dx = e* dx = —e? — (—e"!) = 0.23. « 
1 1 


It is worth mentioning here that a discrete RV can in fact be treated as 
continuous and assigned a corresponding probability density function. If X is a 
discrete RV that takes only the values x1, x2,...,X, with probabilities p1, po,..., Dn 
then we may describe X as a continuous RV with PDF 


f(x) = >> pid(x — xi), (30.44) 
i=1 


where 6(x) is the Dirac delta function discussed in subsection 13.1.3. From (30.42) 
and the fundamental property of the delta function (13.12), we see that 


b 
Pr(a< X <b) = / f(x) dx, 


n b 
-Ynf d(x — xi)dx = S> pi, 
i=1 a 


i 


where the final sum extends over those values of i for which a < x; < b. 


30.4.3 Sets of random variables 


It is common in practice to consider two or more random variables simultane- 
ously. For example, one might be interested in both the height and weight of 
a person drawn at random from a population. In the general case, these vari- 
ables may depend on one another and are described by joint probability density 
functions; these are discussed fully in section 30.11. We simply note here that if 
we have (say) two random variables X and Y then by analogy with the single- 
variable case we define their joint probability density function f(x,y) in such a 
way that, if X and Y are discrete RVs, 


Prix = Xi, Y= yj) = f (Xi, Vj), 
or, if X and Y are continuous RVs, 


Prix << X¥ <x+dx, y< Y <y+dy)=f(x, y)dxdy. 
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In many circumstances, however, random variables do not depend on one 
another, i.e. they are independent. As an example, for a person drawn at random 
from a population, we might expect height and IQ to be independent random 
variables. Let us suppose that X and Y are two random variables with probability 
density functions g(x) and h(y) respectively. In mathematical terms, X and Y are 
independent RVs if their joint probability density function is given by f(x, y) = 
g(x)h(y). Thus, for independent RVs, if X and Y are both discrete then 


Pr(X = xi, Y = yj) = g(xih(y;) 
or, if X and Y are both continuous, then 
Prix < X <x+dx, y< Y < y+dy) = g(x)h(y) dx dy. 


The important point in each case is that the RHS is simply the product of the 
individual probability density functions (compare with the expression for Pr(ANB) 
in (30.22) for statistically independent events A and B). By a simple extension, 
one may also consider the case where one of the random variables is discrete and 
the other continuous. The above discussion may also be trivially extended to any 
number of independent RVs X;, i = 1,2,...,N. 


> The independent random variables X and Y have the PDFs g(x) = e~* and h(y) = 2e~” 
respectively. Calculate the probability that X lies in the interval 1 < X <2 and Y lies in 
the intervalO < Y <1. 


Since X and Y are independent RVs, the required probability is given by 


2 1 
Prl< xX <2,0<Y <1) -| g(x) dx / h(y) dy 
1 0 


2 1 
= / e *dx | 20” dy 
1 0 


[—e™]; x [-e7?"]} = 0.23 x 0.86 = 0.20. < 


30.5 Properties of distributions 


For a single random variable X, the probability density function f(x) contains 
all possible information about how the variable is distributed. However, for the 
purposes of comparison, it is conventional and useful to characterise f(x) by 
certain of its properties. Most of these standard properties are defined in terms 
of averages or expectation values. In the most general case, the expectation value 
E[g(X)] of any function g(X) of the random variable X is defined as 


3; g(x) f(x) for a discrete distribution, 


30.45 
f g(x)f(x)dx for a continuous distribution, ( ) 


E[g(X)] -| 


where the sum or integral is over all allowed values of X. It is assumed that 
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the series is absolutely convergent or that the integral exists, as the case may be. 
From its definition it is straightforward to show that the expectation value has 
the following properties: 


(i) if a is a constant then E[a] = a; 
(i) if a is a constant then E[ag(X)] = aE[g(X)]; 
(ili) if g(X) = s(X) + t(X) then E[g(X)] = E[s(X)] + E[t(Xx)]. 


It should be noted that the expectation value is not a function of X but is 
instead a number that depends on the form of the probability density function 
f(x) and the function g(x). Most of the standard quantities used to characterise 
f(x) are simply the expectation values of various functions of the random variable 
X. We now consider these standard quantities. 


30.5.1 Mean 


The property most commonly used to characterise a probability distribution is 
its mean, which is defined simply as the expectation value E[X] of the variable X 
itself. Thus, the mean is given by 


ELX) = ene for a discrete distribution, (30.46) 


{ xf(x)dx for a continuous distribution. 


The alternative notations y and (x) are also commonly used to denote the mean. 
If in (30.46) the series is not absolutely convergent, or the integral does not exist, 
we say that the distribution does not have a mean, but this is very rare in physical 
applications. 


> The probability of finding a 1s electron in a hydrogen atom in a given infinitesimal volume 
dV is p*p dV, where the quantum mechanical wavefunction yw is given by 


— Aet/%, 


Find the value of the real constant A and thereby deduce the mean distance of the electron 
from the origin. 


Let us consider the random variable R = ‘distance of the electron from the origin’. Since 
the 1s orbital has no 0- or $-dependence (it is spherically symmetric), we may consider 
the infinitesimal volume element dV as the spherical shell with inner radius r and outer 
radius r+dr. Thus, dV = 4nr? dr and the PDF of R is simply 


Pr(ir < R<r+dr)= f(r)dr = 40r’° Ae" dr. 


The value of A is found by requiring the total probability (i.e. the probability that the 
electron is somewhere) to be unity. Since R must lie between zero and infinity, we require 
that 


& [ e7"/% Arr? dr = 1. 
0 
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Integrating by parts we find A = 1/(za3)'/?. Now, using the definition of the mean (30.46), 
we find 


E[R] = | eee / et! dy. 
0 1 Jo 


The integral on the RHS may be integrated by parts and takes the value 3a4/8; conse- 
quently we find that E[R] = 3ao/2. < 


30.5.2 Mode and median 


Although the mean discussed in the last section is the most common measure 
of the ‘average’ of a distribution, two other measures, which do not rely on the 
concept of expectation values, are frequently encountered. 

The mode of a distribution is the value of the random variable X at which the 
probability (density) function f(x) has its greatest value. If there is more than one 
value of X for which this is true then each value may equally be called the mode 
of the distribution. 

The median M of a distribution is the value of the random variable X at which 
the cumulative probability function F(x) takes the value 3 ie. F(M) = ;. Related 
to the median are the lower and upper quartiles Q; and Q, of the PDF, which 
are defined such that 


F(Q))=4,  F(Qu) =?. 


Thus the median and lower and upper quartiles divide the PDF into four regions 
each containing one quarter of the probability. Smaller subdivisions are also 
possible, e.g. the nth percentile, P,,, of a PDF is defined by F(P,,) = n/100. 


> Find the mode of the PDF for the distance from the origin of the electron whose wave- 
function was given in the previous example. 


We found in the previous example that the PDF for the electron’s distance from the origin 
was given by 


2 
f(y= Ar erro, (30.47) 


Differentiating f(r) with respect to r, we obtain 
af By _ TY para, 
dr a ao 


Thus f(r) has turning points at r = 0 and r = ao, where df /dr = 0. It is straightforward 
to show that r = 0 is a minimum and r = ap is a maximum. Moreover, it is also clear that 
r = do is a global maximum (as opposed to just a local one). Thus the mode of f(r) occurs 
atr=do. <4 
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30.5.3 Variance and standard deviation 
The variance of a distribution, V |X], also written a7, is defined by 
DO = uy f (xj) for a discrete distribution, 
f[@- uw)? f(x)dx for a continuous distribution. 


(30.48) 


ris =2 [x7] ={ 


Here u has been written for the expectation value E[X] of X. As in the case of 
the mean, unless the series and the integral in (30.48) converge the distribution 
does not have a variance. From the definition (30.48) we may easily derive the 
following useful properties of V LX]. If a and b are constants then 

(i) V[a] = 0, 

(ii) V[ax + b] = a? V(X]. 


The variance of a distribution is always positive; its positive square root is 
known as the standard deviation of the distribution and is often denoted by o. 
Roughly speaking, o measures the spread (about x = y) of the values that X can 
assume. 


Inserting the expression (30.47) for the PDF f(r) into (30.48), the variance of the random 
variable R is given by 


foe) 4 2 4 foe) 
V[R] = i (r— py gate dr = — i (r4 — 2 +r? pe"! dr, 
0 cry dy Jo 


where the mean = E[R] = 3a)/2. Integrating each term in the integrand by parts we 
obtain 


w 


2 
V[R] = 3a, — 3ua9 +? = a 


Thus the standard deviation of the distribution is ¢ = \/3a)/2. < 


We may also use the definition (30.48) to derive the Bienaymé—Chebyshev 
inequality, which provides a useful upper limit on the probability that random 
variable X takes values outside a given range centred on the mean. Let us consider 
the case of a continuous random variable, for which 


Piix—n2o=f fxd, 


|x—p|>c 
where the integral on the RHS extends over all values of x satisfying the inequality 
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|x — p| > c. From (30.48), we find that 


o> / (x — f(x) dx > c? i f(x) dx. (30.49) 
|x—nl2€ |x—nl2c 
The first inequality holds because both (x — )? and f(x) are non-negative for 
all x, and the second inequality holds because (x — y)* > c? over the range of 
integration. However, the RHS of (30.49) is simply equal to c* Pr(|X — u| > c), 
and thus we obtain the required inequality 


oe 


Pr(|X — pl Sc) < s. 

c 
A similar derivation may be carried through for the case of a discrete random 
variable. Thus, for any distribution f(x) that possesses a variance we have, for 


example, 


Pr([X — p| > 20) < and Pr(|X — p| > 30) < 


1 1 
4 9° 


30.5.4 Moments 


The mean (or expectation) of X is sometimes called the first moment of X, since 
it is defined as the sum or integral of the probability density function multiplied 
by the first power of x. By a simple extension the kth moment of a distribution 
is defined by 


ne EL _ ee (xj) fora ney enn 
{ x“f(x)dx for a continuous distribution. (30.50) 


For notational convenience, we have introduced the symbol jy, to denote E[X*], 
the Ath moment of the distribution. Clearly, the mean of the distribution is then 
denoted by 1, often abbreviated simply to y, as in the previous subsection, as 
this rarely causes confusion. 

A useful result that relates the second moment, the mean and the variance of 
a distribution is proved using the properties of the expectation operator: 


VIX] =E [(X — »)] 


[ 
= E [X?] —2uE[X] +22 
=E [x?] — 22+ 2 
= E [x7] — wv. (30.51) 


(xa) =62)-6) oro = my — 48. 
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By demanding that the sum of the probabilities equals unity we require p = 2/13. Now, 
using the definition of the mean (30.46) for a discrete distribution, 


E[X] = So xf(x)=1x 5p + 2xp + 3xpt+4xp + 5xp + 6x2” 
j 


Bee ee 


ai aE mE 
Similarly, using the definition of the second moment (30.50), 


E[X?] = > xGf( (xj) = x sp +2’p+3p+Hpt+Sp+6 x 2p 
Jj 
_ 253 253 
Sree = 13" 


Finally, using the definition of the variance (30.48), with = 53/13, we obtain 
VIX] = So j— wf) 
j 


= (1-4) 3p + (2— wp + 3 — np t (4-4) pt (5-H) pt (6— Hy 2p 
3120 _ 480 
169° ~ 769" 
It is easy to verify that V[X] = E [X?] —(E[X]). < 
In practice, to calculate the moments of a distribution it is often simpler to use 
the moment generating function discussed in subsection 30.7.2. This is particularly 


true for higher-order moments, where direct evaluation of the sum or integral in 
(30.50) can be somewhat laborious. 


30.5.5 Central moments 


The variance V[X] is sometimes called the second central moment of the distribu- 
tion, since it is defined as the sum or integral of the probability density function 
multiplied by the second power of x — yu. The origin of the term ‘central’ is that by 
subtracting x from x before squaring we are considering the moment about the 
mean of the distribution, rather than about x = 0. Thus the kth central moment 
of a distribution is defined as 


ae [0x = ' 7 ee jj — wf (x;) for a discrete distribution, (30.52) 


[@- u)‘f(x)dx for a continuous distribution. 


It is convenient to introduce the notation vy, for the kth central moment. Thus 
V[X] = v2 and we may write (30.51) as v2 = py — pi. Clearly, the first central 
moment of a distribution is always zero since, for example in the continuous case, 


n= fox—wpeode =f xpeydx—a f foydx=n—Wx1)=0. 
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We note that the notation 4, and v, for the moments and central moments 
respectively is not universal. Indeed, in some books their meanings are reversed. 

We can write the kth central moment of a distribution in terms of its kth and 
lower-order moments by expanding (X — 1)‘ in powers of X. We have already 
noted that v2 = i — yj, and similar expressions may be obtained for higher-order 
central moments. For example, 


v3 = E [(X —y)°] 
= E [X* — 3X? + 3uX — ui] 
= os — 3ypo + 3yjin — Hy 
= ps — 3p + 2uh, (30.53) 


In general, it is straightforward to show that 


Vie = pe —* Cr pagar Fe + (HD! * Crp re Fe + (ADEN EO Ca — Dp. 
(30.54) 


Once again, direct evaluation of the sum or integral in (30.52) can be rather 
tedious for higher moments, and it is usually quicker to use the moment generating 
function (see subsection 30.7.2), from which the central moments can be easily 
evaluated as well. 


>The PDF for a Gaussian distribution (see subsection 30.9.1) with mean « and variance 


2 


o° is given by 


f(x) = 


1 (=F 
aaa exp I- 72 é 


Obtain an expression for the kth central moment of this distribution. 


As an illustration, we will perform this calculation by evaluating the integral in (30.52) 
directly. Thus, the kth central moment of f(x) is given by 


ae i. "(= WEFO) de 


00 


1 is k (x— ) 
xX — pl)" ex dx 
o/2n [i sd | 20? 
1 a y 
= Par _vexp| a5 dy, (30.55) 


where in the last line we have made the substitution y = x — uw. It is clear that if k is 
odd then the integrand is an odd function of y and hence the integral equals zero. Thus, 
vy = 0 if k is odd. When k is even, we could calculate v, by integrating by parts to obtain 
a reduction formula, but it is more elegant to consider instead the standard integral (see 
subsection 6.4.2) 


I [ew ay?) dy = ng, 


00 
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and differentiate it repeatedly with respect to « (see section 5.12). Thus, we obtain 
dl = 
don i: y exp(—ay”) dy = ie elle 

aI 


ai -| y* exp(—ay’) dy = ($)(3)a?a0? 


d"I ice} A455 
Se TD" f yexpl—ay?dy = INE) Qn tye arene 


Setting « = 1/(207) and substituting the above result into (30.55), we find (for k even) 


= GG) Gk — Qe? = (13) (k- Dok. 


One may also characterise a probability distribution f(x) using the closely 
related normalised and dimensionless central moments 


From this set, y3 and y4 are more commonly called, respectively, the skewness 
and kurtosis of the distribution. The skewness y3 of a distribution is zero if it is 
symmetrical about its mean. If the distribution is skewed to values of x smaller 
than the mean then 73 < 0. Similarly y3; > 0 if the distribution is skewed to higher 
values of x. 

From the above example, we see that the kurtosis of the Gaussian distribution 
(subsection 30.9.1) is given by 


y 


It is therefore common practice to define the excess kurtosis of a distribution 
as y4 — 3. A positive value of the excess kurtosis implies a relatively narrower 
peak and wider wings than the Gaussian distribution with the same mean and 
variance. A negative excess kurtosis implies a wider peak and shorter wings. 

Finally, we note here that one can also describe a probability density function 
f(x) in terms of its cumulants, which are again related to the central moments. 
However, we defer the discussion of cumulants until subsection 30.7.4, since their 
definition is most easily understood in terms of generating functions. 


30.6 Functions of random variables 


Suppose X is some random variable for which the probability density function 
f(x) is known. In many cases, we are more interested in a related random variable 
Y = Y(X), where Y(X) is some function of X. What is the probability density 
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function g(y) for the new random variable Y? We now discuss how to obtain 
this function. 


30.6.1 Discrete random variables 


If X is a discrete RV that takes only the values x;, i = 1,2,...,n, then Y must 
also be discrete and takes the values y; = Y(x;), although some of these values 
may be identical. The probability function for Y is given by 


aly) = i Fo) Hy= yy (30.56) 


0 otherwise, 


where the sum extends over those values of j for which y; = Y(x;). The simplest 
case arises when the function Y (X) possesses a single-valued inverse X(Y ). In this 
case, only one x-value corresponds to each y-value, and we obtain a closed-form 
expression for g(y) given by 


_ Jf) ify=y. 
g(y) = 


0) otherwise. 


If Y(X) does not possess a single-valued inverse then the situation is more 
complicated and it may not be possible to obtain a closed-form expression for 
g(y). Nevertheless, whatever the form of Y(X), one can always use (30.56) to 
obtain the numerical values of the probability function g(y) at y = yj. 


30.6.2 Continuous random variables 
If X is a continuous RV, then so too is the new random variable Y = Y(X). The 


probability that Y lies in the range y to y + dy is given by 


g(y)dy = | : F(x) dx, (30.57) 


where dS corresponds to all values of x for which Y lies in the range y to y+ dy. 
Once again the simplest case occurs when Y (X) possesses a single-valued inverse 
X(Y). In this case, we may write 


x(y+dy) ae ; x(y)+| # dy a . 
g(y)dy = / f(x‘) dx -| f(x’) dx’, 
x(y) x(y) 
from which we obtain 
: dx 
e(y) = f(xy) rll (30.58) 
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lighthouse 


ii beam 


O coastline 


Figure 30.8 The illumination of a coastline by the beam from a lighthouse. 


>A lighthouse is situated at a distance L from a straight coastline, opposite a point O, and 
sends out a narrow continuous beam of light simultaneously in opposite directions. The beam 
rotates with constant angular velocity. If the random variable Y is the distance along the 
coastline, measured from O, of the spot that the light beam illuminates, find its probability 
density function. 


The situation is illustrated in figure 30.8. Since the light beam rotates at a constant angular 
velocity, @ is distributed uniformly between —z/2 and 2/2, and so f(0) = 1/x. Now 
y = Ltan0, which possesses the single-valued inverse 0 = tan~'(y/L), provided that 0 lies 
between —1/2 and 2/2. Since dy/d0 = Lsec? 0 = L(1 + tan?0) = L[1 + (y/L)’], from 
(30.58) we find 


1 
T 


do 
dy 


1 
nLfl + (y/L)’] 


A distribution of this form is called a Cauchy distribution and is discussed in subsec- 
tion 30.9.5. < 


g(y) 


for —-a <y<o. 


If Y(X) does not possess a single-valued inverse then we encounter complica- 
tions, since there exist several intervals in the X-domain for which Y lies between 
y and y+dy. This is illustrated in figure 30.9, which shows a function Y(X) 
such that X(Y) is a double-valued function of Y. Thus the range y to y + dy 
corresponds to X’s being either in the range x; to x; + dx; or in the range x2 to 
x2 + dx. In general, it may not be possible to obtain an expression for g(y) in 
closed form, although the distribution may always be obtained numerically using 
(30.57). However, a closed-form expression may be obtained in the case where 
there exist single-valued functions x;(y) and x2(y) giving the two values of x that 
correspond to any given value of y. In this case, 


x1(y+dy) ; x2(y+dy) ; 
sv)dy=|f soya] +] [soar 
xi(y) x2(y) 
from which we obtain 
: dx : dx 
av) = feeuly) || + Fa) | >]. (30.59) 
y y 
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y+dy 


dx; dx x 


Figure 30.9 Illustration of a function Y (X) whose inverse X(Y ) is a double- 
valued function of Y. The range y to y+dy corresponds to X being either in 
the range x; to x; + dx, or in the range x2 to x2 + dx). 


This result may be generalised straightforwardly to the case where the range y to 
y + dy corresponds to more than two x-intervals. 


> The random variable X is Gaussian distributed (see subsection 30.9.1) with mean and 
variance o. Find the PDF of the new variable Y = (X — p)*/o?. 


It is clear that X(Y) is a double-valued function of Y. However, in this case, it is 
straightforward to obtain single-valued functions giving the two values of x that correspond 
to a given value of y; these are x; = u—o./y and x» =yw+ a. /Y, where Jy is taken to 
mean the positive square root. 

The PDF of X is given by 


1 D 
fx) = = ee| a 


Since dx,/dy = —a/(2,/9) and dx/dy = o/(2,/¥), from (30.59) we obtain 


0) = —exp(-$)] 4] + ett) | 57 
iad o/2n Nea 2/9|  o/2n Pwra? 2/9 
1 = 
= 7a 2) se exp(—4y). 


As we shall see in subsection 30.9.3, this is the gamma distribution (4, 5). < 


30.6.3 Functions of several random variables 


We may extend our discussion further, to the case in which the new random 
variable is a function of several other random variables. For definiteness, let us 
consider the random variable Z = Z(X,Y), which is a function of two other 
RVs X and Y. Given that these variables are described by the joint probability 
density function f(x,y), we wish to find the probability density function p(z) of 
the variable Z. 
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If X and Y are both discrete RVs then 
pz) => f(xy), (30.60) 
ij 
where the sum extends over all values of i and j for which Z(x;, y;) = z. Similarly, 
if X and Y are both continuous RVs then p(z) is found by requiring that 


p(z)dz = If. f(x, y) dx dy, (30.61) 


where dS is the infinitesimal area in the xy-plane lying between the curves 
Z(x,y) =z and Z(x,y) =z+dz. 


> Suppose X and Y are independent continuous random variables in the range —« to o, 
with PDFs g(x) and h(y) respectively. Obtain expressions for the PDFs of Z = X +Y and 
W=XY. 


Since X and Y are independent RVs, their joint PDF is simply f(x,y) = g(x)h(y). Thus, 
from (30.61), the PDF of the sum Z = X + Y is given by 


oe) z+dz—x 
peydz= f dx go) [dy hw 


00 zZ—X 


a (/- g(x)h(z — x) ax) dz. 


Thus p(z) is the convolution of the PDFs of g and h (ie. p = g * h, see subsection 13.1.7). 
In a similar way, the PDF of the product W = XY is given by 


ioe) (w+dw)/|x| 
aw) dw = / dx g(x) / dy hy) 


co »/\x| 


re dx 
= cs g(x)h(w/x) =) dw < 


The prescription (30.61) is readily generalised to functions of n random variables 
Z = Z(X1,X2,...,Xn), in which case the infinitesimal ‘volume’ element dS is the 
region in X1X2--+X,-space between the (hyper)surfaces Z (x1, X2,...,Xn) = z and 
Z(X1,X2,---,Xn) = 2Z+dz. In practice, however, the integral is difficult to evaluate, 
since one is faced with the complicated geometrical problem of determining the 
limits of integration. Fortunately, an alternative (and powerful) technique exists 
for evaluating integrals of this kind. One eliminates the geometrical problem by 
integrating over all values of the variables x; without restriction, while shifting 
the constraint on the variables to the integrand. This is readily achieved by 
multiplying the integrand by a function that equals unity in the infinitesimal 
region dS and zero elsewhere. From the discussion of the Dirac delta function in 
subsection 13.1.3, we see that 6(Z (x1, X2,...;Xn)—Z) dz satisfies these requirements, 
and so in the most general case we have 


pte) = fi foe fer, xa..-s3n) (Ze %0...5%4) 2) dda dy 
(30.62) 
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where the range of integration is over all possible values of the variables x;. This 
integral is most readily evaluated by substituting in (30.62) the Fourier integral 
representation of the Dirac delta function discussed in subsection 13.1.4, namely 


1 Sartre : 
O(Z(X1,X2,--.5Xn) —Z) = al eik(Z(X1X20%n)—2) (30.63) 
T =X) 
This is best illustrated by considering a specific example. 


>A general one-dimensional random walk consists of n independent steps, each of which 
can be of a different length and in either direction along the x-axis. If g(x) is the PDF for 


the (positive or negative) displacement X along the x-axis achieved in a single step, obtain 
an expression for the PDF of the total displacement S after n steps. 


The total displacement S is simply the algebraic sum of the displacements X; achieved in 
each of the n steps, so that 


SSX IG, 


Since the random variables are independent and have the same PDF g(x), their joint 
PDF is simply g(x1)g(x2)--- g(Xn). Substituting this into (30.62), together with (30.63), we 


obtain 
w= [- [- [a 8(X1)g(X2)-- a0) 5 — | erat dle died > dy 
ee —iks s ikx i 
-=f- dke (fax Je x ; (30.64) 


It is convenient to define the characteristic function C(k) of the variable X as 


c= | “glade, 


which is simply related to the Fourier transform of g(x). Then (30.64) may be written as 
(s) = : / < eS [C(k)]" dk 
PS) = 55 oe 


Thus p(s) can be found by evaluating two Fourier integrals. Characteristic functions will 
be discussed in more detail in subsection 30.7.3. < 


30.6.4 Expectation values and variances 


In some cases, one is interested only in the expectation value or the variance 
of the new variable Z rather than in its full probability density function. For 
definiteness, let us consider the random variable Z = Z(X, Y ), which is a function 
of two RVs X and Y with a known joint distribution f(x,y); the results we will 
obtain are readily generalised to more (or fewer) variables. 

It is clear that E[Z] and V[Z] can be obtained, in principle, by first using the 
methods discussed above to obtain p(z) and then evaluating the appropriate sums 
or integrals. The intermediate step of calculating p(z) is not necessary, however, 
since it is straightforward to obtain expressions for E[Z] and V[Z] in terms of 
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the variables X and Y. For example, if X and Y are continuous RVs then the 
expectation value of Z is given by 


E[Z] = | :vt2)ae = [ [ Zesnfessavdy, (30.65) 


An analogous result exists for discrete random variables. 

Integrals of the form (30.65) are often difficult to evaluate. Nevertheless, we 
may use (30.65) to derive an important general result concerning expectation 
values. If X and Y are any two random variables and a and b are arbitrary 
constants then by letting Z = aX + bY we find 


E[aX + bY] =aE[X] + bE[Y]. 


Furthermore, we may use this result to obtain an approximate expression for the 
expectation value E[Z(X, Y )] of any arbitrary function of X and Y. Letting uy = 
E[X] and wy = E[Y], and provided Z(X, Y) can be reasonably approximated by 
the linear terms of its Taylor expansion about the point (wx, uy), we have 


a) 


OZ OZ 
ZX.) = Ziaur) + (SE) x wot (SE) Or Ly), 


(30.66) 


where the partial derivatives are evaluated at X = zy and Y = py. Taking the 
expectation values of both sides, we find 
i 


E[Z(X,Y)] = Ziuxatr+ (SS (ELXI-ux+ (F 


SS) (BLY -1v) = Zier 
oY 


which gives the approximate result E[Z(X, Y)] + Z(uy, py). 
By analogy with (30.65), the variance of Z = Z(X, Y) is given by 


vig) = fe—nz)n(e)dz = ff 20%») nF fs) dxdy, 
(30.67) 


where zz = E[Z]. We may use this expression to derive a second useful result. If 
X and Y are two independent random variables, so that f(x,y) = g(x)h(y), and 
a, b and c are constants then by setting Z = aX + bY +c in (30.67) we obtain 


V[aX +bY +c] =@V[X] + BV [Y]. (30.68) 


From (30.68) we also obtain the important special case 


VIX +Y] =V[X —Y] =V[X]4+V[Y]. 


Provided X and Y are indeed independent random variables, we may obtain 
an approximate expression for V[Z(X,Y)], for any arbitrary function Z(X, Y), 
in a similar manner to that used in approximating E[Z(X, Y)] above. Taking the 
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variance of both sides of (30.66), and using (30.68), we find 


B 2 2 
V[Z(x,Y)] = (=) V[X]+ (=) V[Y], (30.69) 


the partial derivatives being evaluated at X = wy and Y = py. 


30.7 Generating functions 


As we saw in chapter 16, when dealing with particular sets of functions fy, 
each member of the set being characterised by a different non-negative integer 
n, it is sometimes possible to summarise the whole set by a single function of a 
dummy variable (say t), called a generating function. The relationship between 
the generating function and the nth member f;, of the set is that if the generating 
function is expanded as a power series in t then f, is the coefficient of t”. For 
example, in the expansion of the generating function G(z,t) = (1 — 2zt + ?)-1/?, 
the coefficient of t” is the nth Legendre polynomial P,,(z), i.e. 


G(z,t) = (1—22t +P)? = S0 Palz)t". 
n=0 
We found that many useful properties of, and relationships between, the members 
of a set of functions could be established using the generating function and other 
functions obtained from it, e.g. its derivatives. 

Similar ideas can be used in the area of probability theory, and two types of 
generating function can be usefully defined, one more generally applicable than 
the other. The more restricted of the two, applicable only to discrete integral 
distributions, is called a probability generating function; this is discussed in the 
next section. The second type, a moment generating function, can be used with 
both discrete and continuous distributions and is considered in subsection 30.7.2. 
From the moment generating function, we may also construct the closely re- 
lated characteristic and cumulant generating functions; these are discussed in 
subsections 30.7.3 and 30.7.4 respectively. 


30.7.1 Probability generating functions 


As already indicated, probability generating functions are restricted in applicabil- 
ity to integer distributions, of which the most common (the binomial, the Poisson 
and the geometric) are considered in this and later subsections. In such distribu- 
tions a random variable may take only non-negative integer values. The actual 
possible values may be finite or infinite in number, but, for formal purposes, 
all integers, 0,1,2,... are considered possible. If only a finite number of integer 
values can occur in any particular case then those that cannot occur are included 
but are assigned zero probability. 
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If, as previously, the probability that the random variable X takes the value x, 


is f(X,), then 
Yo fn) = 1. 


In the present case, however, only non-negative integer values of x, are possible, 
and we can, without ambiguity, write the probability that X takes the value n as 
fr» with 


si =, (30.70) 


n=0 


We may now define the probability generating function ® x(t) by 


®x(t) = ‘ fot”. (30.71) 


n=0 
It is immediately apparent that ®y(t) = E[t*] and that, by virtue of (30.70), 
®y(1) =1. 
Probably the simplest example of a probability generating function (PGF) is 
provided by the random variable X defined by 


‘ if the outcome of a single trial is a ‘success’, 


0 if the trial ends in ‘failure’. 
If the probability of success is p and that of failure g (= 1 — p) then 
y(t) = qt? + pt! +0+04+---=q+pt. (30.72) 


This type of random variable is discussed much more fully in subsection 30.8.1. 
In a similar but slightly more complicated way, a Poisson-distributed integer 
variable with mean 2 (see subsection 30.8.4) has a PGF 


ox()=>>* tt = eel (30.73) 


We note that, as required, ®y(1) = 1 in both cases. 

Useful results will be obtained from this kind of approach only if the summation 
(30.71) can be carried out explicitly in particular cases and the functions derived 
from ®x(t) can be shown to be related to meaningful parameters. Two such 
relationships can be obtained by differentiating (30.71) with respect to t. Taking 
the first derivative we find 


id co co 
: a = Do nfat | = Oy(1) = So nfs = ELX), (30.74) 


n=0 n=0 
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and differentiating once more we obtain 


2 ee) foe) 
ox) =Sintn-1fit"? > Of) = So nn— If, = E[X(X — 1). 
a ae n=O (30.75) 


Equation (30.74) shows that ®,(1) gives the mean of X. Using both (30.75) and 
(30.51) allows us to write 
(1) + @ (1) — [Oy (1)]? = EEX(X — 1] + EDX] - (EXT? 
= E [X*| — E[X] + E[X] — (E[X]) 
= E [X?] —(E[X]? 
= V[X], (30.76) 


and so express the variance of X in terms of the derivatives of its probability 
generating function. 


>A random variable X is given by the number of trials needed to obtain a first success 
when the chance of success at each trial is constant and equal to p. Find the probability 
generating function for X and use it to determine the mean and variance of X. 


Clearly, at least one trial is needed, and so fy = 0. If n (=> 1) trials are needed for the first 
success, the first n — 1 trials must have resulted in failure. Thus 


Pr(X =n) = q""'p, n>1, (30.77) 


where q = 1 — p is the probability of failure in each individual trial. 
The corresponding probability generating function is thus 


y(t) => fat" = (ay 


n=0 n=1 


_P qt pt 
by t)" 30.78 
(qt) Teg = i=at (30.78) 


n=1 


where we have used the result for the sum of a geometric series, given in chapter 4, to 
obtain a closed-form expression for ®y(t). Again, as must be the case, ®y(1) = 1. 

To find the mean and variance of X we need to evaluate ®4,(1) and ®{(1). Differentiating 
(30.78) gives 


‘ P ; pi 
Oy, (t) = => O(1)=S=-, 
y(t) Gan? (1) moe 
ny — 29 nay — 2PQ _ 2q 
O(t) = (1-40? = O()=— = pe 
Thus, using (30.74) and (30.76), 
E[X] = ®,(1) = 7 


V[X] = ©£(1) + ®L(1) — [© 
_2q 1 1 sq 
p 2° 


(LP 


PoP? 

A distribution with probabilities of the general form (30.77) is known as a geometric 
distribution and is discussed in subsection 30.8.2. This form of distribution is common in 
‘waiting time’ problems (subsection 30.9.3). < 


1159 


PROBABILITY 


r 


Figure 30.10 The pairs of values of n and r used in the evaluation of Oy+y (t). 


Sums of random variables 


We now turn to considering the sum of two or more independent random 
variables, say X and Y, and denote by S) the random variable 


S=X+Y. 


If s(t) is the PGF for S, the coefficient of t” in its expansion is given by the 
probability that X + Y =n and is thus equal to the sum of the probabilities that 
X =r and Y =n—r for all values of r in 0 <r <n. Since such outcomes for 
different values of r are mutually exclusive, we have 


Pr(X¥ + Y =n)= Ne Pr(X =r) Pr(Y =n—r). (30.79) 
r=0 
Multiplying both sides of (30.79) by t” and summing over all values of n enables 
us to express this relationship in terms of probability generating functions as 
follows: 


Oy, y(t) = So PX +¥=n=> >) P(X =r) PY =n—r)"” 


n=0 n=0 r=0 
=S0 > P(X =r)? Pr(¥ =n—r). 
r=0 n=r 


The change in summation order is justified by reference to figure 30.10, which 
illustrates that the summations are over exactly the same pairs of values of n and 
r, but with the first (inner) summation over the points in a column rather than 
over the points in a row. Now, setting n =r +s gives the final result, 


®x y(t) = So Prix =r)t" ss Pr(Y = s)t* 
r=0 s=0 
= Ox(t)®y (t), (30.80) 
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ie. the PGF of the sum of two independent random variables is equal to the 
product of their individual PGFs. The same result can be deduced in a less formal 
way by noting that if X and Y are independent then 


E (e+!) =E [7] E(t"). 


Clearly result (30.80) can be extended to more than two random variables by 
writing $3 = S.+Z etc., to give 


Os" x) = [os t), (30.81) 


and, further, if all the X; have the same probability distribution, 
D5", x) = [Ox(o]”. (30.82) 


This latter result has immediate application in the deduction of the PGF for the 
binomial distribution from that for a single trial, equation (30.72). 


Variable-length sums of random variables 


As a final result in the theory of probability generating functions we show how to 
calculate the PGF for a sum of N random variables, all with the same probability 
distribution, when the value of N is itself a random variable but one with a 
known probability distribution. In symbols, we wish to find the distribution 
of 


Se a ie, (30.83) 


where N is a random variable with Pr(N = n) = h, and PGF yn(t) = 


SS hyt". 
The probability €, that Sy =k is given by a sum of conditional probabilities, 
namely 


fe = > Pr(N =n) Pr(Xo + Xi + Xo +--+ + Xn =k) 
n=0 


a Ss h, X coefficient of t* in [®x(t)]”. 
n=0 


Multiplying both sides of this equation by t* and summing over all k, we obtain 


S Formally Xo = 0 has to be included, since Pr(N = 0) may be non-zero. 
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an expression for the PGF 3s(t) of Sy: 
Es(t)= So &t* = SOAS ~h, x coefficient of t* in [Ox (x)]" 
k=0 k=0  n=0 


oe) oe) 


a Se hy, yy t' x coefficient of t in [®y(t)]" 
n=0 k=0 


= ST hnlbx(o) 


n=0 
= yn(®x(t)). (30.84) 


In words, the PGF of the sum Sy is given by the compound function yy(®x(t)) 
obtained by substituting ®x(t) for t in the PGF for the number of terms N in 
the sum. We illustrate this with the following example. 


> The probability distribution for the number of eggs in a clutch is Poisson distributed with 
mean 4, and the probability that each egg will hatch is p (and is independent of the size of 
the clutch). Use the results stated in (30.72) and (30.73) to show that the PGF (and hence 
the probability distribution) for the number of chicks that hatch corresponds to a Poisson 
distribution having mean Ap. 


The number of chicks that hatch is given by a sum of the form (30.83) in which X; = 1 if 
the ith chick hatches and X; = 0 if it does not. As given by (30.72), ®x(t) is thus (1—p)+ pt. 
The value of N is given by a Poisson distribution with mean /; thus, from (30.73), in the 
terminology of our previous discussion, 


in(t) = ee. 
We now substitute these forms into (30.84) to obtain 
Es(t) = exp(—A) exp[Ax(t)] 
= exp(—A) exp{A[(1 — p) + pt]} 
= exp(—/p) exp(Apt). 
But this is exactly the PGF of a Poisson distribution with mean Ap. 
That this implies that the probability is Poisson distributed is intuitively obvious since, 
in the expansion of the PGF as a power series in ¢, every coefficient will be precisely 


that implied by such a distribution. A solution of the same problem by direct calculation 
appears in the answer to exercise 30.29. < 


30.7.2 Moment generating functions 


As we saw in section 30.5 a probability function is often expressed in terms of 
its moments. This leads naturally to the second type of generating function, a 
moment generating function. For a random variable X, and a real number t, the 
moment generating function (MGF) is defined by 


My(t) = E [e*] ef (xi) for a discrete distribution, 
= e = 
- fe®f(x)dx for a continuous distribution. (30.85) 
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The MGF will exist for all values of t provided that X is bounded and always 
exists at the point t = 0 where M(0) = E(1) = 1. 

It will be apparent that the PGF and the MGF for a random variable X 
are closely related. The former is the expectation of t* whilst the latter is the 
expectation of e*: 


Ox(t)=E[t*],  Mx(t)=E[e*]. 


The MGF can thus be obtained from the PGF by replacing t by e', and vice 
versa. The MGF has more general applicability, however, since it can be used 
with both continuous and discrete distributions whilst the PGF is restricted to 
non-negative integer distributions. 

As its name suggests, the MGF is particularly useful for obtaining the moments 
of a distribution, as is easily seen by noting that 


2y2 
E [e*] meli+ex+ So+ | 


2, 
HLF ERX HED] Ste, 


Assuming that the MGF exists for all t around the point t = 0, we can deduce 
that the moments of a distribution are given in terms of its MGF by 


d" Mx(t) 


E[x"] = 4 


(30.86) 


t=0 


Similarly, by substitution in (30.51), the variance of the distribution is given by 
V[X] = M%(0) — [M4(0)]’, (30.87) 


where the prime denotes differentiation with respect to t. 


> The MGF for the Gaussian distribution (see the end of subsection 30.9.1) is given by 
Myx(t) = exp (ut + 30°C’) ; 


Find the expectation and variance of this distribution. 


Using (30.86), 
M(t) = (+. 0°t) exp (ut + 407#7) => ELX])=My(0)=4, 
MY (t) = [o? + (w+ 0°t)’] exp (ut + $070) => MZ(0)=07 +2. 
Thus, using (30.87), 
V[XJH=0 4 -—p =o". 


That the mean is found to be w and the variance o? justifies the use of these symbols in 
the Gaussian distribution. < 


The moment generating function has several useful properties that follow from 
its definition and can be employed in simplifying calculations. 
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Scaling and shifting 


If Y =aX +b, where a and b are arbitrary constants, then 
My (t)=E [e’] =E [e*™)] = cE [e**] =e Mx(at). (30.88) 


This result is often useful for obtaining the central moments of a distribution. If the 
MFG of X is My(t) then the variable Y = X—y has the MGF My (t) = e“My(t), 
which clearly generates the central moments of X, ie. 


qd" 


dt" 


E[(X — 1") = ty") = M0) ( l-™Mx(0) 


t=0 


Sums of random variables 


If X1, Xo,..., Xn are independent random variables and Sy = X,;+X.+---+Xy 
then 


N 
Ms, (t) =E [e"| =E pe eeea eat =E HI oxi 
i=1 


Since the X; are independent, 


N 


N 
Ms,(t) = [JE [e*'] = [[ Mx,(0. (30.89) 
i=1 


i=1 


In words, the MGF of the sum of N independent random variables is the product 
of their individual MGFs. By combining (30.89) with (30.88), we obtain the more 
general result that the MGF of Sy = c1X1 +c2X2 +---+cyXy (where the c; are 
constants) is given by 


N 
Ms, (t) =] Mxi(cit). (30.90) 
it 


Variable-length sums of random variables 


Let us consider the sum of N independent random variables X; (i = 1,2,...,.N), all 
with the same probability distribution, and let us suppose that N is itself a random 
variable with a known distribution. Following the notation of section 30.7.1, 


Sy =X, +X. +---+ Xn, 


where N is a random variable with Pr(N = n) = h, and probability generating 
function yy(t) = >> h,t". For definiteness, let us assume that the X; are continuous 
RVs (an analogous discussion can be given in the discrete case). Thus, the 
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probability that value of Sy lies in the interval s to s +ds is given by’ 
Pr(s < Sy < s+ds) = S> Priv =n)Pr(s < Xo + X1 + X2--- + Xn < s4ds). 
n=0 


Write Pr(s < Sy <s+ds) as f(s) ds and Pr(s < Xo + X; + X2--- + X, < s4ds) 
as f,(s) ds. The kth moment of the PDF f(s) is given by 


w= J te0 ds = Js 5 PrN = n)f ,(s) ds 


n=0 


= SPW = n) [shu ds 


n=0 


= So My x (k!x coefficient of t* in [Mx(t)]") 
n=0 


Thus the MGF of Sy is given by 


Ms, (t) = ot = Sohn So t* x coefficient of e* in [Mx(0)]" 
k=0 n=0 k=0 
= So in{Mx(0)]" 
n=0 
= xn(Mx(t)). 


In words, the MGF of the sum Sy is given by the compound function yv(Myx(t)) 
obtained by substituting My(t) for t in the PGF for the number of terms N in 
the sum. 


Uniqueness 


If the MGF of the random variable X, is identical to that for X, then the 
probability distributions of X; and X> are identical. This is intuitively reasonable 
although a rigorous proof is complicated,’ and beyond the scope of this book. 


30.7.3. Characteristic function 


The characteristic function (CF) of a random variable X is defined as 


Cx(t) =E [e"*] = 


oF e"if(x;) for a discrete distribution, 


fe®f(x)dx for a continuous distribution (30.91) 


S As in the previous section, Xo has to be formally included, since Pr(N = 0) may be non-zero. 


1 See, for example, P. A. Moran, An Introduction to Probability Theory (New York: Oxford Science 
Publications, 1984). 
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so that Cx(t) = My(it), where My(t) is the MGF of X. Clearly, the characteristic 
function and the MGF are very closely related and can be used interchangeably. 
Because of the formal similarity between the definitions of Cx(t) and My(t), the 
characteristic function possesses analogous properties to those listed in the previ- 
ous section for the MGF, with only minor modifications. Indeed, by substituting it 
for t in any of the relations obeyed by the MGF and noting that Cy(t) = Mx(it), 
we obtain the corresponding relationship for the characteristic function. Thus, for 
example, the moments of X are given in terms of the derivatives of Cx(t) by 


E[X"] = (—i)"C¥(0). 


Similarly, if Y = aX + b then Cy(t) = e!”'Cy(at). 

Whether to describe a random variable by its characteristic function or by its 
MGF is partly a matter of personal preference. However, the use of the CF does 
have some advantages. Most importantly, the replacement of the exponential e'* 
in the definition of the MGF by the complex oscillatory function e* in the CF 
means that in the latter we avoid any difficulties associated with convergence of 
the relevant sum or integral. Furthermore, when X is a continous RV, we see 
from (30.91) that Cx(t) is related to the Fourier transform of the PDF f(x). As 
a consequence of Fourier’s inversion theorem, we may obtain f(x) from Cx(t) by 
performing the inverse transform 


CO 


f(x) = x Cx(t)e™ dt. 


—oo 


30.7.4 Cumulant generating function 


As mentioned at the end of subsection 30.5.5, we may also describe a probability 
density function f(x) in terms of its cumulants. These quantities may be expressed 
in terms of the moments of the distribution and are important in sampling theory, 
which we discuss in the next chapter. The cumulants of a distribution are best 
defined in terms of its cumulant generating function (CGF), given by Kx(t) = 
In Mxy(t) where Mx(t) is the MGF of the distribution. If Kx(t) is expanded as a 
power series in t then the kth cumulant x, of f(x) is the coefficient of t*/k!: 


2. 3 


Ky(t) = In My(t) St + ms is Ka reer (30.92) 


Since Mx(0) = 1, Kx(t) contains no constant term. 


> Find all the cumulants of the Gaussian distribution discussed in the previous example. 


The moment generating function for the Gaussian distribution is My(t) = exp (ut + 30°). 
Thus, the cumulant generating function has the simple form 


Ky(t) = In My(t) = wt + 500". 
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Comparing this expression with (30.92), we find that x: = pu, K2 = o? and all other 
cumulants are equal to zero. < 


We may obtain expressions for the cumulants of a distribution in terms of its 
moments by differentiating (30.92) with respect to t to give 


dKx 1 dMyx 


dt My dt 


Expanding each term as power series in t and cross-multiplying, we obtain 


te ? ? 
(es tmrtieg +) (14m tog +) = (utmrtingt), 


and, on equating coefficients of like powers of t on each side, we find 


My = Ki, 
fg =k2+ Ki 14, 
H3 = 3 + 2K2M1 + Ki fb, 


ba = K4 + 3«3 M1 + 3k2 M2 + K1 p13, 


fe = Ke FAN Cy eat, He FC Fo Et 
Solving these equations for the «x, we obtain (for the first four cumulants) 
Ki = [|M1, 
K2 = [2 — Ht = V2, 
K3 = M3 — 3y2p1 + 2u} = v3, 
Kea = Mg — 4yizyy + L2pout — 35 — Out = v4 — 393. (30.93) 


Higher-order cumulants may be calculated in the same way but become increas- 
ingly lengthy to write out in full. 

The principal property of cumulants is their additivity, which may be proved 
by combining (30.92) with (30.90). If X1, Xo, ..., Xy are independent random 
variables and Ky,(t) for i = 1,2,...,N is the CGF for X; then the CGF of 
Sn = CX, +0.X. +-+-+cnyXy (where the c; are constants) is given by 


N 
Ks,(t) = > Kx,(cit). 
i=l 
Cumulants also have the useful property that, under a change of origin X > 
X +a the first cumulant undergoes the change x; — x; +a but all higher-order 
cumulants remain unchanged. Under a change of scale X — bX, cumulant x, 
undergoes the change x, — b'k,. 
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Distribution Probability law f(x) MGF E[xX] V [xX] 


binomial "Copq"* (pe' + q)" np npq 
es ial IC pig Pp rq rq 
negative binomia p'q Cer ae a P 
: e pe! 1 q 
t x—1 
geometric gp age ? P 
3 (Ng) \n\(N—n)! N=n 
hypergeometric eee np N-1 npq 
Poisson <¢ ele) A A 
x! 


Table 30.1 Some important discrete probability distributions. 


30.8 Important discrete distributions 


Having discussed some general properties of distributions, we now consider the 
more important discrete distributions encountered in physical applications. These 
are discussed in detail below, and summarised for convenience in table 30.1; we 
refer the reader to the relevant section below for an explanation of the symbols 
used. 


30.8.1 The binomial distribution 


Perhaps the most important discrete probability distribution is the binomial dis- 
tribution. This distribution describes processes that consist of a number of inde- 
pendent identical trials with two possible outcomes, A and B = A. We may call 
these outcomes ‘success’ and ‘failure’ respectively. If the probability of a success 
is Pr(A) = p then the probability of a failure is Pr(B) = q = 1 — p. If we perform 
n trials then the discrete random variable 


X = number of times A occurs 


can take the values 0, 1,2,...,n; its distribution amongst these values is described 
by the binomial distribution. 

We now calculate the probability that in n trials we obtain x successes (and so 
n—x< failures). One way of obtaining such a result is to have x successes followed 
by n—x failures. Since the trials are assumed independent, the probability of this is 

pp--PX gq---q =prgh™. 
K—— —SSS— 
x times n—x times 


This is, however, just one permutation of x successes and n—x failures. The total 
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f(x) f(x) 
n=5,p=0.6 n=5, p= 0.167 
0.4 0.47 
0.3 0.3 
0.2+ 0.2+ 
0.14 0.1 
0 | Qe ys 
012345 x 012345 x 
f(x) f(x) 
n= 10, p=0.6 n= 10, p = 0.167 
0.4 0.47 
0.3- 0.37 
0.2 0.2+ 
0.17 | | 0.1 
0 | . 0 | . 
012345678910 x 012345678910 x 


Figure 30.11 Some typical binomial distributions with various combinations 
of parameters n and p. 


number of permutations of n objects, of which x are identical and of type 1 and 
n—x are identical and of type 2, is given by (30.33) as 
n! asf 
x'(n—x)! Oe 
Therefore, the total probability of obtaining x successes from n trials is 
f(x) = Pr(X = x) = "Cy pg" = "Cy pL = py", (30.94) 


which is the binomial probability distribution formula. When a random variable 
X follows the binomial distribution for n trials, with a probability of success p, 
we write X ~ Bin(n,p). Then the random variable X is often referred to as a 
binomial variate. Some typical binomial distributions are shown in figure 30.11. 


If a single six-sided die is rolled five times, what is the probability that a six is thrown 


exactly three times? 


Here the number of ‘trials’ n = 5, and we are interested in the random variable 
X = number of sixes thrown. 


Since the probability of a ‘success’ is p = te the probability of obtaining exactly three sixes 
in five throws is given by (30.94) as 


5! 1\? /5\° 
Pr(X = 3) se—a (3) (3) 0.032. < 
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For evaluating binomial probabilities a useful result is the binomial recurrence 


formula 
P(X =x+1)=" () Pr(X = x), (30.95) 
q\x+1 
which enables successive probabilities Pr(X¥ = x +k), k = 1,2,..., to be calculated 


once Pr(X = x) is known; it is often quicker to use than (30.94). 


> The random variable X is distributed as X ~ Bin(3, 3). Evaluate the probability function 
f(x) using the binomial recurrence formula. 


The probability Pr(X = 0) may be calculated using (30.94) and is 
Pr(X = 0) = °Co (3)" (5)° = & 


The ratio p/q = 3/4 = 1 in this case and so, using the binomial recurrence formula 
(30.95), we find 


30) L 3 
Pr(X = 1) =1x > xe =5, 
=. 3. 3 
Pr(X =2)=1x 2 x 5 = 5, 
32) 3° 
Pr(X =3)=1x > x = 5, 


results which may be verified by direct application of (30.94). < 


We note that, as required, the binomial distribution satifies 


n 


S_ f@) = 55°C, pq" * = (pt ay =. 
x=0 x=0 


Furthermore, from the definitions of E[X] and V[X] for a discrete distribution, 
we may show that for the binomial distribution E[X] = np and V[X] = npq. The 
direct summations involved are, however, rather cumbersome and these results 
are obtained much more simply using the moment generating function. 


The moment generating function for the binomial distribution 


To find the MGF for the binomial distribution we consider the binomial random 
variable X to be the sum of the random variables X;, i = 1,2,...,n, which are 
defined by 


e if a ‘success’ occurs on the ith trial, 
L 


0 if a ‘failure’ occurs on the ith trial. 
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Thus 
M(t) = E [e™] = e™ x Pr(X; = 0) +e" x Pr(X; = 1) 
=1xqtexp 
=pe'+q. 
From (30.89), it follows that the MGF for the binomial distribution is given by 
M(t)= II Mit) = (pe' +)". (30.96) 
i=l 
We can now use the moment generating function to derive the mean and 
variance of the binomial distribution. From (30.96) 
M'(t) = npe'(pe! +g)", 
and from (30.86) 
E[X] = M'(0) = np(p +4)""| = np, 


where the last equality follows from p+ q = 1. 
Differentiating with respect to t once more gives 


M"(t) = e'(n — I)np*(pel + q)"* + e'np(pe! +g)", 
and from (30.86) 
E[X?] = M"(0) = n?p? — np? + np. 
Thus, using (30.87) 
V[X] = M"(0) — [M'(0)|” = Wp? — np? + np — Wp? = np(1 — p) = npa. 


Multiple binomial distributions 


Suppose X and Y are two independent random variables, both of which are 
described by binomial distributions with a common probability of success p, but 
with (in general) different numbers of trials nm; and m, so that X ~ Bin(m, p) 
and Y ~ Bin(n, p). Now consider the random variable Z = X + Y. We could 
calculate the probability distribution of Z directly using (30.60), but it is much 
easier to use the MGF (30.96). 

Since X and Y are independent random variables, the MGF Mz(t) of the new 
variable Z = X + Y is given simply by the product of the individual MGFs 
Mx(t) and My(t). Thus, we obtain 


Mz(t) = Mx(t)My (t) = (pe' +g)" (pe +g)" = (pe +q)""™, 


which we recognise as the MGF of Z ~ Bin(n, +m, p). Hence Z is also described 
by a binomial distribution. 
This result may be extended to any number of binomial distributions. If X;, 
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i= 1,2,...,N, is distributed as X; ~ Bin(n;,p) then Z = X; + X. +---+ Xn is 
distributed as Z ~ Bin(n) +n2+---+ny,p), as would be expected since the result 
of >, n; trials cannot depend on how they are split up. A similar proof is also 
possible using either the probability or cumulant generating functions. 

Unfortunately, no equivalent simple result exists for the probability distribution 
of the difference Z = X — Y of two binomially distributed variables. 


30.8.2 The geometric and negative binomial distributions 


A special case of the binomial distribution occurs when instead of the number of 
successes we consider the discrete random variable 


X = number of trials required to obtain the first success. 


The probability that x trials are required in order to obtain the first success, is 
simply the probability of obtaining x — 1 failures followed by one success. If the 
probability of a success on each trial is p, then for x > 0 


f(x) = Pr(X =x)=(1 py 'p = gp, 


where q = 1 — p. This distribution is sometimes called the geometric distribution. 
The probability generating function for this distribution is given in (30.78). By 
replacing t by e’ in (30.78) we immediately obtain the MGF of the geometric 
distribution 


t 


pe 


M(t) = ——— 
(t) 1 — qe’ 


from which its mean and variance are found to be 
1 q 
E[xX]=-, VIX] =<. 
Dp Pp 


Another distribution closely related to the binomial is the negative binomial 
distribution. This describes the probability distribution of the random variable 


X = number of failures before the rth success. 


One way of obtaining x failures before the rth success is to have r — 1 successes 
followed by x failures followed by the rth success, for which the probability is 
PPe--P  X 44-94 X p= PG. 
—— —— 
r—l1times x times 


However, the first r+ x — 1 factors constitute just one permutation of r— 1 
successes and x failures. The total number of permutations of these r+ x—1 
objects, of which r — 1 are identical and of type 1 and x are identical and of type 
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2, is "*'C,. Therefore, the total probability of obtaining x failures before the 
rth success is 


f(x) = P(X =x) =" CpG, 


which is called the negative binomial distribution (see the related discussion on 
p. 1137). It is straightforward to show that the MGF of this distribution is 


m= (qa) 


and that its mean and variance are given by 


E(xj= and vex} =. 
D D 


30.8.3 The hypergeometric distribution 


In subsection 30.8.1 we saw that the probability of obtaining x successes in n 
independent trials was given by the binomial distribution. Suppose that these n 
‘trials’ actually consist of drawing at random n balls, from a set of N such balls 
of which M are red and the rest white. Let us consider the random variable 
X = number of red balls drawn. 

On the one hand, if the balls are drawn with replacement then the trials are 
independent and the probability of drawing a red ball is p = M/N each time. 
Therefore, the probability of drawing x red balls in n trials is given by the 
binomial distribution as 


n ! *1 py)’. 


BSS aaa 


On the other hand, if the balls are drawn without replacement the trials are not 
independent and the probability of drawing a red ball depends on how many red 
balls have already been drawn. We can, however, still derive a general formula 
for the probability of drawing x red balls in n trials, as follows. 

The number of ways of drawing x red balls from M is C,, and the number 
of ways of drawing n— x white balls from N — M is ‘-“C,_,. Therefore, the 
total number of ways to obtain x red balls in n trials is “C, N-@C,_,. However, 
the total number of ways of drawing n objects from N is simply “C,,. Hence the 
probability of obtaining x red balls in n trials is 


M N-—M 
Cy Cr_x 


Pr(X = x)= NG 
_ M! (N—M)! n\(N —n)! 
x!(M — x)! (n—x)\(N -—M—n+x)! N! ? O97) 
(Np)\(Nq)!n!(N —n)! (30.98) 


x\(Np — x)\(n—x)'"(Nq —n+x)!NV 
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where in the last line p = M/N and q = 1 —p. This is called the hypergeometric 
distribution. 

By performing the relevant summations directly, it may be shown that the 
hypergeometric distribution has mean 


E[X] =n 


vn 
wR 


and variance 


nM(N—M)(N—n) _N—n 


Nl NAN—-1) N—1 


>In the UK National Lottery each participant chooses six different numbers between 1 
and 49. In each weekly draw six numbered winning balls are subsequently drawn. Find the 


probabilities that a participant chooses 0,1,2,3,4,5,6 winning numbers correctly. 


The probabilities are given by a hypergeometric distribution with N (the total number of 
balls) = 49, M (the number of winning balls drawn) = 6, and n (the number of numbers 
chosen by each participant) = 6. Thus, substituting in (30.97), we find 


Fae) So oe Rely a i. a 
pray= “GES =, mas) = Geo 
pr(6) = 6 1 


C6 13.98 x 106° 


It can easily be seen that 
6 
So Pr(i) = 0.44 + 0.41 + 0.13 + 0.02 + O10) = 1, 
i=0 


as expected. < 


Note that if the number of trials (balls drawn) is small compared with N, M 
and N —M then not replacing the balls is of little consequence, and we may 
approximate the hypergeometric distribution by the binomial distribution (with 
p = M/N); this is much easier to evaluate. 


30.8.4 The Poisson distribution 


We have seen that the binomial distribution describes the number of successful 
outcomes in a certain number of trials n. The Poisson distribution also describes 
the probability of obtaining a given number of successes but for situations 
in which the number of ‘trials’ cannot be enumerated; rather it describes the 
situation in which discrete events occur in a continuum. Typical examples of 
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discrete random variables X described by a Poisson distribution are the number 
of telephone calls received by a switchboard in a given interval, or the number 
of stars above a certain brightness in a particular area of the sky. Given a mean 
rate of occurrence / of these events in the relevant interval or area, the Poisson 
distribution gives the probability Pr(X = x) that exactly x events will occur. 

We may derive the form of the Poisson distribution as the limit of the binomial 
distribution when the number of trials n — oo and the probability of ‘success’ 
p — 0, in such a way that np = A remains finite. Thus, in our example of a 
telephone switchboard, suppose we wish to find the probability that exactly x 
calls are received during some time interval, given that the mean number of calls 
in such an interval is 4. Let us begin by dividing the time interval into a large 
number, n, of equal shorter intervals, in each of which the probability of receiving 
a call is p. As we let n > « then p — 0, but since we require the mean number 
of calls in the interval to equal 2, we must have np = A. The probability of x 
successes in n trials is given by the binomial formula as 


n! ; 
*(1 — py", (30.99) 


ES a OIL, 


Now as n > ©, with x finite, the ratio of the n-dependent factorials in (30.99) 
behaves asymptotically as a power of n, ie. 


n! 


= lim n(n— 1)(n—2)---(n—x+1)~n". 


1 
noo (n _ x)! no 


Also 


es om . A= py! P e? 
Ai—X joan 
im ae —p)*= _ Tap ne aT 


Thus, using 2 = np, (30.99) tends to the Poisson distribution 


e7? Ax 


f(x) =PXX =x) = 


(30.100) 


which gives the probability of obtaining exactly x calls in the given time interval. 
As we shall show below, 4 is the mean of the distribution. Events following a 
Poisson distribution are usually said to occur randomly in time. 

Alternatively we may derive the Poisson distribution directly, without consid- 
ering a limit of the binomial distribution. Let us again consider our example 
of a telephone switchboard. Suppose that the probability that x calls have been 
received in a time interval t is P,(t). If the average number of calls received in a 
unit time is 2 then in a further small time interval At the probability of receiving 
a call is AAt, provided At is short enough that the probability of receiving two or 
more calls in this small interval is negligible. Similarly the probability of receiving 
no call during the same small interval is simply 1 — AAt. 

Thus, for x > 0, the probability of receiving exactly x calls in the total interval 
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t+ At is given by 
P(t + At) = P,(t)(1 — AAt) + Py_1(t)AAt. 


Rearranging the equation, dividing through by At and letting At — 0, we obtain 
the differential recurrence equation 
aP,(t) 
dt 
For x = 0 (ie. no calls received), however, (30.101) simplifies to 


dPo(t) 
dt 


which may be integrated to give Po(t) = Po(0)e~““. But since the probability Po(0) 
of receiving no calls in a zero time interval must equal unity, we have Po(t) = e~”’. 
This expression for Po(t) may then be substituted back into (30.101) with x = 1 
to obtain a differential equation for P,(t) that has the solution P;(t) = Ate~“. 
We may repeat this process to obtain expressions for P2(t), P3(t),..., Px(t), and we 
find 


= AP,_;(t) — AP,(0). (30.101) 


= —/Po(t), 


Aty* 
P(t) = ( oe". (30.102) 
x! 
By setting t = 1 in (30.102), we again obtain the Poisson distribution (30.100) for 
obtaining exactly x calls in a unit time interval. 
If a discrete random variable is described by a Poisson distribution of mean / 


then we write X ~ Po(J). As it must be, the sum of the probabilities is unity: 


ice) x 


ss Pr(X =x)=e% 2 “4 =e =1, 
x=0 y 


x=0 


From (30.100) we may also derive the Poisson recurrence formula, 


= rr Pr(X =x) for x =0,1,2,..., 
i (30.103) 


which enables successive probabilities to be calculated easily once one is known. 


>A person receives on average one e-mail message per half-hour interval. Assuming that 


the e-mails are received randomly in time, find the probabilities that in any particular hour 
0, 1, 2, 3, 4, 5 messages are received. 


Let X = number of e-mails received per hour. Clearly the mean number of e-mails per 
hour is two, and so X follows a Poisson distribution with 2 = 2, Le. 


xX 


Pr(X = x) = Me, 
x! 
Thus Pr(X = 0) = e~? = 0.135, Pr(X = 1) = 2e? = 0.271, Pr(X = 2) = 27e7/2! = 0.271, 


Pr(X = 3) = 23e7/3! = 0.180, Pr(X = 4) = 24e77/4! = 0.090, Pr(X = 5) = 25e7/5! = 
0.036. These results may also be calculated using the recurrence formula (30.103). < 
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f(x) f(x) 
A=1 = 
0.3 4 0.3 4 ae 
0.2 + 0.2 4 
0.1 + 0.1 5 
0 0 | 
012345 x: 01234567 x 
f(x) 
03 at 
02 
0.1 
0 | Lo 
01234567891011 x 


Figure 30.12 Three Poisson distributions for different values of the parame- 


ter 2. 


The above example illustrates the point that a Poisson distribution typically 
rises and then falls. It either has a maximum when x is equal to the integer part 
of 4 or, if 2 happens to be an integer, has equal maximal values at x = 4—1 and 
x = 4. The Poisson distribution always has a long ‘tail’ towards higher values of X 
but the higher the value of the mean the more symmetric the distribution becomes. 
Typical Poisson distributions are shown in figure 30.12. Using the definitions of 
mean and variance, we may show that, for the Poisson distribution, E[X] = / and 
V [X] = A. Nevertheless, as in the case of the binomial distribution, performing 
the relevant summations directly is rather tiresome, and these results are much 
more easily proved using the MGF. 


The moment generating function for the Poisson distribution 


The MGF of the Poisson distribution is given by 


Oo welt —A 4x ek jet)* ee a 
My(t) =E [e*| _ ee e Ss _ ae ( _ =e = ele —1) 
ane, ent (30.104) 
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from which we obtain 


MY (t) = (Ae + dele), 
Thus, the mean and variance of the Poisson distribution are given by 


E[X]=M(0)=2 = and ~~ V[X] = M0) — [M,(0)? = 2. 


The Poisson approximation to the binomial distribution 


Earlier we derived the Poisson distribution as the limit of the binomial distribution 
when n — oo and p > 0 in such a way that np = 4 remains finite, where / is the 
mean of the Poisson distribution. It is not surprising, therefore, that the Poisson 
distribution is a very good approximation to the binomial distribution for large 
n (=> 50, say) and small p (< 0.1, say). Moreover, it is easier to calculate as it 
involves fewer factorials. 


>In a large batch of light bulbs, the probability that a bulb is defective is 0.5%. For a 
sample of 200 bulbs taken at random, find the approximate probabilities that 0, 1 and 2 of 


the bulbs respectively are defective. 


Let the random variable X = number of defective bulbs in a sample. This is distributed 
as X ~ Bin(200, 0.005), implying that 2 = np = 1.0. Since n is large and p small, we may 
approximate the distribution as X ~ Po(1), giving 


1* 
Pr(X =x) xe! = 
x! 


from which we find Pr(X = 0) = 0.37, Pr(X = 1) = 0.37, Pr(X = 2) = 0.18. For comparison, 
it may be noted that the exact values calculated from the binomial distribution are identical 
to those found here to two decimal places. < 


Multiple Poisson distributions 


Mirroring our discussion of multiple binomial distributions in subsection 30.8.1, 
let us suppose X and Y are two independent random variables, both of which 
are described by Poisson distributions with (in general) different means, so that 
X ~ Po(j,;) and Y ~ Po(Az). Now consider the random variable Z = X + Y. We 
may calculate the probability distribution of Z directly using (30.60), but we may 
derive the result much more easily by using the moment generating function (or 
indeed the probability or cumulant generating functions). 

Since X and Y are independent RVs, the MGF for Z is simply the product of 
the individual MGFs for X and Y. Thus, from (30.104), 


Mz(t) = Mx(t)My (t) = etl V pall) = ofr tAa)(e! ay 


which we recognise as the MGF of Z ~ Po(A; + 42). Hence Z is also Poisson 
distributed and has mean 4, + 22. Unfortunately, no such simple result holds for 
the difference Z = X — Y of two independent Poisson variates. A closed-form 
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expression for the PDF of this Z does exist, but it is a rather complicated 
combination of exponentials and a modified Bessel function 


> Two types of e-mail arrive independently and at random: external e-mails at a mean rate 
of one every five minutes and internal e-mails at a rate of two every five minutes. Calculate 


the probability of receiving two or more e-mails in any two-minute interval. 


X = number of external e-mails per two-minute interval, 
Y = number of internal e-mails per two-minute interval. 
Since we expect on average one external e-mail and two internal e-mails every five minutes 


we have X ~ Po(0.4) and Y ~ Po(0.8). Letting Z = X + Y we have Z ~ Po(0.4+ 0.8) = 
Po(1.2). Now 


Pr(Z > 2) =1—Pr(Z <2) =1—Pr(Z =0)— Pr(Z = 1) 


and 
Pr(Z = 0) =e7!? = 0.301, 
_ git = 0.361. 


Hence Pr(Z > 2) = 1 — 0.301 — 0.361 = 0.338. < 


The above result can be extended, of course, to any number of Poisson processes, 
so that if X; = Po(/;), i = 1,2,...,n then the random variable Z = X; + X) + 
--- +X, is distributed as Z ~ Po(A; +A. +--- +A). 


30.9 Important continuous distributions 


Having discussed the most commonly encountered discrete probability distri- 
butions, we now consider some of the more important continuous probability 
distributions. These are summarised for convenience in table 30.2; we refer the 
reader to the relevant subsection below for an explanation of the symbols used. 


30.9.1 The Gaussian distribution 


By far the most important continuous probability distribution is the Gaussian 
or normal distribution. The reason for its importance is that a great many 
random variables of interest, in all areas of the physical sciences and beyond, are 
described either exactly or approximately by a Gaussian distribution. Moreover, 
the Gaussian distribution can be used to approximate other, more complicated, 
probability distributions. 


S For a derivation see, for example, M. P. Hobson and A. N. Lasenby, Monthly Notices of the Royal 
Astronomical Society, 298, 905 (1998). 
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Distribution Probability law f(x) MGF E[Xx] V [xX] 
2 
Gaussian = exp ( =! exp(ut+ 4070?) o 
. a A 1 1 
exponential de € — ;) Fi Z 
A r—1 j—Ax A ; 
gamma ro” e € — ;) = RZ 
i ¢ ! (n/2)-1 ,—x/2 ! va 
chi-squared PT (n/D)* e (; — x) n 2n 
uniform : oe eae al ee 
b-a (b—a)t 2 12 


Table 30.2 Some important continuous probability distributions. 


The probability density function for a Gaussian distribution of a random 
variable X, with mean E[X] = pu and variance VX] = o?, takes the form 


Ache oh _lexrp 2 
f(y= =| ( ) |. (30.105) 


The factor 1/,/2z arises from the normalisation of the distribution, 


” pOodx = 1; 


the evaluation of this integral is discussed in subsection 6.4.2. The Gaussian 
distribution is symmetric about the point x = yu and has the characteristic ‘bell’ 
shape shown in figure 30.13. The width of the curve is described by the standard 
deviation o: if o is large then the curve is broad, and if o is small then the curve 
is narrow (see the figure). At x = w+o, f(x) falls to e~!/? = 0.61 of its peak 
value; these points are points of inflection, where d*f /dx? = 0. When a random 
variable X follows a Gaussian distribution with mean yu and variance o*, we write 
X ~ N(u,0?). 

The effects of changing yw and o are only to shift the curve along the x-axis or 
to broaden or narrow it, respectively. Thus all Gaussians are equivalent in that 
a change of origin and scale can reduce them to a standard form. We therefore 
consider the random variable Z = (X — u)/o, for which the PDF takes the form 


1 2 
z) = ——exp|-—-— }, 30.106 
H)= p( =) (30.106) 
which is called the standard Gaussian distribution and has mean yp = 0 and 


variance o* = 1. The random variable Z is called the standard variable. 
From (30.105) we can define the cumulative probability function for a Gaussian 
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u=3 
0.4 
la=1 
\ 
0.3 | 
i 
| Mend 
lan\ 
02+ ff | \\ 


Figure 30.13 The Gaussian or normal distribution for mean p = 3 and 
various values of the standard deviation o. 


$(z) 


#n 


—4 —2 0 a 2 


Figure 30.14 On the left, the standard Gaussian distribution $(z); the shaded 
area gives Pr(Z < a) = ®(a). On the right, the cumulative probability function 
@(z) for a standard Gaussian distribution $(z). 


distribution as 


F(x) = Pr(X < x)= = [ exp -5 (—*)| du, 


Oo 


(30.107) 


where u is a (dummy) integration variable. Unfortunately, this (indefinite) integral 
cannot be evaluated analytically. It is therefore standard practice to tabulate val- 
ues of the cumulative probability function for the standard Gaussian distribution 


(see figure 30.14), Le. 


Zz 2 
on smgeqe =) ep (-5) du. (30.108) 
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It is usual only to tabulate ®(z) for z > 0, since it can be seen easily, from 
figure 30.14 and the symmetry of the Gaussian distribution, that ®(—z) = 1—®(z); 
see table 30.3. Using such a table it is then straightforward to evaluate the 
probability that Z lies in a given range of z-values. For example, for a and b 
constant, 


Pr(Z < a) = D(a), 
Pr(Z > a) = 1— Pa), 
Pr(a < Z < b) = O(b) — Da). 


Remembering that Z = (X — u)/o and comparing (30.107) and (30.108), we see 
that 


F(x) = 0 (=—*), 


Oo 


and so we may also calculate the probability that the original random variable 
X lies in a given x-range. For example, 


b 
Pra<X <b) = — = | exp -3 (4) du (30.109) 
= F(b) — F(a) (30.110) 
er eae —o(—*). (30.111) 
oO oO 


p> If X is described by a Gaussian distribution of mean pw and variance o?, calculate the 
probabilities that X lies within 1o, 2c and 30 of the mean. 


From (30.111) 


Pr(u — no < X < w+no) = O(n) — O(—n) = O(n) — [1 — O(n)], 


and so from table 30.3 


Pr(u—o < X <p+o) = 20(1) — 1 = 0.6826 = 68.3%, 
Pr(u— 20 < X < +20) = 20(2) — 1 = 0.9544 = 95.4%, 
Pr(u— 30 < X < w+ 3) = 20(3) — 1 = 0.9974 = 99.7%. 


Thus we expect X to be distributed in such a way that about two thirds of the values will 
lie between p —o and w+o, 95% will lie within 2¢ of the mean and 99.7% will lie within 
30 of the mean. These limits are called the one-, two- and three-sigma limits respectively; 
it is particularly important to note that they are independent of the actual values of the 
mean and variance. 4 


There are many other ways in which the Gaussian distribution may be used. 
We now illustrate some of the uses in more complicated examples. 
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.00 O1 02 .03 04 .05 .06 07 .08 .09 


5000 =.5040 =.5080)=— 5120 .5160) 5199) 5239. 5279 5319 5359 
5398 5438) 5478 = 5517) 5557) .5596) 5636) 567557145753 
5793 5832) .5871 = 5910-5948 5987 = 6026 = 6064 =— 6103S 6141 
6179 6217 = .6255. 6293 6331 = 6368 )=— 6406 = 6443 6480) .6517 
6554 .6591 6628 .6664 .6700 .6736 .6772 .6808 .6844 .6879 


6915 .6950 .6985 .7019 .7054 7088) .7123, 715771907224 
7257 7291 7324 735773897422, .7454.— 7486) £7517 .7549 
.7580 .7611 = .7642)—s 7673. 7704S .7734 7764 .7794 £7823 .7852 
7881  .7910 .7939 .7967 £7995 .8023.— 8051 — 8078 ~=— 8106 ~—.8 133 
8159 8186 =.8212) 8238) = 8264) 828983158340 = 8365 £8389 


8413 8438 =. 8461) = 84858508) = 8531) 85548577 8599 8621 
8643 8665 8686 .8708 .8729 .8749 .8770 .8790 =.8810 8830 
8849 .8869 8888  .8907 .8925 .8944 .8962 .8980 .8997 .9015 
9032 9049 9066 §=.9082 9099) 9115 9131) 9147) 91629177 
9192 9207 9222 9236 §=.9251 = .9265 = 9279) 9292, 9306 ~— 9319 


9332 9345 9357 9370 = 9382 9394 9406 = 9418 =—.9429 9441 
9452 9463 9474 9484 9495 9505 .9515  .9525  .9535  .9545 
9554 9564 =.9573, 95829591 .9599 9608) = 9616) 96259633 
9641 9649 9656 .9664 .9671 .9678 .9686 .9693 .9699 .9706 
9713, 9719 9726 9732, 9738 974497509756) 9761 9767 


9772 9778 9783-9788 = 97939798 ~— 9803 9808 = 98129817 
9821 .9826 9830 .9834 9838 =.9842, 9846 = 9850 .9854 9857 
9861 .9864 9868 .9871 9875 .9878 9881 .9884 .9887 .9890 
9893 9896 §=.9898 9901 = 9904. 9906) 9909) 9911 = 9913 9916 
9918 9920 .9922 9925 9927 .9929 .9931 .9932 9934 .9936 


9938 9940 9941 .9943 .9945 .9946 .9948 .9949 9951 .9952 
9953, 9955 9956 9957 9959 9960S 9961) 9962 9963 9964 
9965 9966 9967 .9968 .9969 .9970 .9971 .9972 9973 .9974 
9974 9975 9976 9977 9977 .9978 .9979 .9979 9980  .9981 
9981 .9982 9982 .9983 .9984 .9984 9985 .9985 .9986 .9986 


9987 9987 9987 .9988 .9988 .9989 .9989 .9989 .9990 .9990 
9990 9991 9991 9991 9992 9992 9992 9992 99939993 
9993 9993 9994 9994 9994 9994 9994 999599959995 
9995 9995 9995 .9996 .9996 .9996 .9996 .9996 9996 .9997 
9997 9997 9997 9997 9997 .9997 9997 .9997 9997  .9998 


Table 30.3 The cumulative probability function ®(z) for the standard Gaus- 
sian distribution, as given by (30.108). The units and the first decimal place 
of z are specified in the column under ®(z) and the second decimal place is 
specified by the column headings. Thus, for example, ®(1.23) = 0.8907. 
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> Sawmill A produces boards whose lengths are Gaussian distributed with mean 209.4 cm 
and standard deviation 5.0 cm. A board is accepted if it is longer than 200 cm but is 
rejected otherwise. Show that 3% of boards are rejected. 

Sawmill B produces boards of the same standard deviation but of mean length 210.1 cm. 
Find the proportion of boards rejected if they are drawn at random from the outputs of A 
and B in the ratio 3:1. 


Let X = length of boards from A, so that X ~ N(209.4, (5.0)*) and 


200—\ _ © 200 — 209.4 
0 > 5.0 


Pr(X < 200) = ® ( ) = (1.88). 


But, since ®(—z) = 1 — ®(z) we have, using table 30.3, 
Pr(X < 200) = 1 — ®(1.88) = 1 — 0.9699 = 0.0301, 


ie. 3.0% of boards are rejected. 
Now let Y = length of boards from B, so that Y ~ N(210.1, (5.0)*) and 


Pr(Y < 200) = erro = 0(—2.02) 
= 1— (2.02) 


= 1— 0.9783 = 0.0217. 


Therefore, when taken alone, only 2.2% of boards from B are rejected. If, however, boards 
are drawn at random from A and B in the ratio 3 : 1 then the proportion rejected is 


+3 x 0.030 + 1 x 0.022) = 0.028 = 2.8%. < 


We may sometimes work backwards to derive the mean and standard deviation 
of a population that is known to be Gaussian distributed. 


> The time taken for a computer ‘packet’ to travel from Cambridge UK to Cambridge MA 
is Gaussian distributed. 6.8% of the packets take over 200 ms to make the journey, and 
3.0% take under 140 ms. Find the mean and standard deviation of the distribution. 


Let X = journey time in ms; we are told that X ~ N(,07) where p and o are unknown. 
Since 6.8% of journey times are longer than 200 ms, 


Pr(X > 200) =1—® (=) = 0.068, 


from which we find 


© (= “) = 1 — 0.068 = 0.932. 


Using table 30.3, we have therefore 
200 — yw 


= 1.49. (30.112) 


Also, 3.0% of journey times are under 140 ms, so 


140 — 
o 


Pr(X < 140) =0® ( ) = 0.030. 
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Now using ®(—z) = 1 — ®(z) gives 


® (HK) = 1 — 0.030 = 0.970. 


Using table 30.3 again, we find 


ue = 188. (30.113) 


Solving the simultaneous equations (30.112) and (30.113) gives w = 173.5, o = 17.8. < 


The moment generating function for the Gaussian distribution 
Using the definition of the MGF (30.85), 


Mx(t)=E [e*| — i : exp [os b =f) dx 


«© 6/20 
=cexp(yt+ 4o7t’), 


where the final equality is established by completing the square in the argument 
of the exponential and writing 


ro wl [x—(ut ot)? 
= NE ae 
? he o./2n exp 202 as 


However, the final integral is simply the normalisation integral for the Gaussian 
distribution, and so c = 1 and the MGF is given by 


Mx(t) = exp (ut + 3078") . (30.114) 


We showed in subsection 30.7.2 that this MGF leads to ELX] = p and V[X] =o’, 
as required. 


Gaussian approximation to the binomial distribution 


We may consider the Gaussian distribution as the limit of the binomial distribu- 
tion when the number of trials n > 00 but the probability of a success p remains 
finite, so that np — « also. (This contrasts with the Poisson distribution, which 
corresponds to the limit n > o and p > 0 with np = 4 remaining finite.) In 
other words, a Gaussian distribution results when an experiment with a finite 
probability of success is repeated a large number of times. We now show how 
this Gaussian limit arises. 
The binomial probability function gives the probability of x successes in n trials 

as 

% n! x n—-X 

f(x) = xin—xylP (1—p)"™. 


Taking the limit as n — oo (and x > ©) we may approximate the factorials by 
Stirling’s approximation 


naa (E) 


1185 


PROBABILITY 


f(x) (binomial) — f(x) (Gaussian) 


x 
0 0.0001 0.0001 
1 0.0016 0.0014 
2 0.0106 0.0092 
3 0.0425 0.0395 
4 0.1115 0.1119 
5 0.2007 0.2091 
6 0.2508 0.2575 
7 0.2150 0.2091 
8 0.1209 0.1119 
9 0.0403 0.0395 

10 0.0060 0.0092 


Table 30.4 Comparison of the binomial distribution for n = 10 and p = 0.6 
with its Gaussian approximation. 


to obtain 


f(x) 1 es @ _ aoe *(l— py 


n n 
x 


=FRor| (x +4) In=— (n—x+4)In 


n—-Xx 


n n 


+ xInp + (n—x)In(1—p)). 


By expanding the argument of the exponential in terms of y = x — np, where 
1<y <np and keeping only the dominant terms, it can be shown that 


Taner 1 1 1 (x —npy 
I) aA” | el. 
which is of Gaussian form with » = np and o = ,/np(1— p). 

Thus we see that the value of the Gaussian probability density function f(x) is 
a good approximation to the probability of obtaining x successes in n trials. This 
approximation is actually very good even for relatively small n. For example, if 
n= 10 and p = 0.6 then the Gaussian approximation to the binomial distribution 
is (30.105) with u = 10 x 0.6 = 6 and o = ,/10 x 0.6(1—0.6) = 1.549. The 
probability functions f(x) for the binomial and associated Gaussian distributions 
for these parameters are given in table 30.4, and it can be seen that the Gaussian 
approximation is a good one. 

Strictly speaking, however, since the Gaussian distribution is continuous and 
the binomial distribution is discrete, we should use the integral of f(x) for the 
Gaussian distribution in the calculation of approximate binomial probabilities. 
More specifically, we should apply a continuity correction so that the discrete 
integer x in the binomial distribution becomes the interval [x — 0.5, x + 0.5] in 
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the Gaussian distribution. Explicitly, 


Pr(X = x) =e | o[-3 5 (—*) i du. 


The Gaussian approximation is particularly useful for estimating the binomial 
probability that X lies between the (integer) values x; and xp, 


a se 2 
Pr(xy < X < x2) & xo |- aC Hy" du. 


of2n Jx;-05 2 0 


>A manufacturer makes computer chips of which 10% are defective. For a random sample 
of 200 chips, find the approximate probability that more than 15 are defective. 


We first define the random variable 
X = number of defective chips in the sample, 


which has a binomial distribution X ~ Bin(200,0.1). Therefore, the mean and variance of 
this distribution are 


E[X]=200x01=20 and  V[X]=200x 01x (1—0.1) = 18, 


and we may approximate the binomial distribution with a Gaussian distribution such that 
X ~ N(20, 18). The standard variable is 


and so, using X = 15.5 to allow for the continuity correction, 


eet = Pr(Z > —1.06) 
Vis 


= Pr(Z < 1.06) = 0.86. < 


Pr(X > 15.5) =Pr (z > 


Gaussian approximation to the Poisson distribution 


We first met the Poisson distribution as the limit of the binomial distribution for 
n — co and p — 0, taken in such a way that np = 4 remains finite. Further, in 
the previous subsection, we considered the Gaussian distribution as the limit of 
the binomial distribution when n — oo but p remains finite, so that np — © also. 
It should come as no surprise, therefore, that the Gaussian distribution can also 
be used to approximate the Poisson distribution when the mean / becomes large. 
The probability function for the Poisson distribution is 


Ax 


fx) =e4S 


xP? 


which, on taking the logarithm of both sides, gives 


In f(x) =A +xIn2—Inx!. (30.115) 
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Stirling’s approximation for large x gives 
x\x 
xt ee (2nx (2) 
implying that 
Inx! = In J2nx +xInx — x 
which, on substituting into (30.115), yields 
In f(x)  —A + xInA—(xInx — x) — In V2nx. 


Since we expect the Poisson distribution to peak around x = /, we substitute 
€ =x—/A to obtain 


Inf(x)~ 2+ (2+6){Ind n|2(1+5)]}+G+e)—-In Vn +6). 


Using the expansion In(1 + z) = z —2?/24+---, we find 


Inf) we— +a (5 —) - inn -(§-5) 


A 222 2A2 


2 
a 5 — In /2nd, 


when only the dominant terms are retained, after using the fact that e is of the 
order of the standard deviation of x, ie. of order 2!/?. On exponentiating this 
result we obtain 


f(x) ® 


1 (x — A)? 
Orr | om | : 
which is the Gaussian distribution with = A and o? = A. 

The larger the value of 4, the better is the Gaussian approximation to the 
Poisson distribution; the approximation is reasonable even for 4 = 5, but 2 > 10 
is safer. As in the case of the Gaussian approximation to the binomial distribution, 
a continuity correction is necessary since the Poisson distribution is discrete. 


We first define the random variable 


X = number of messages received in a day. 


Thus E[X] = 1 x 24 = 24, and so X ~ Po(24). Since 2 > 10 we may approximate the 
Poisson distribution by X ~ N(24, 24). Now the standard variable is 


xX — 24 
Z =——, 
(24 
and, using the continuity correction, we find 
23.5 — =) 


Ju 


= Pr(Z > —0.102) = Pr(Z < 0.102) = 0.54. « 


Pr(X > 23.5) = Pr (z > 
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In fact, almost all probability distributions tend towards a Gaussian when the 
numbers involved become large — that this should happen is required by the 
central limit theorem, which we discuss in section 30.10. 


Multiple Gaussian distributions 


Suppose X and Y are independent Gaussian-distributed random variables, so 
that X ~ N(u1,07) and Y ~ N(,03). Let us now consider the random variable 
Z = X+/Y. The PDF for this random variable may be found directly using 
(30.61), but it is easier to use the MGF. From (30.114), the MGFs of X and Y 
are 

My(t) = exp (jut + 4070’), My (t) = exp (pot + 403t") . 


Using (30.89), since X and Y are independent RVs, the MGF of Z = X + Y is 
simply the product of Mx(t) and My(t). Thus, we have 


Mz(t) = Mx(t)My (t) = exp (uit + t07t”) exp (pat + 3030") 
= exp [(mi + wt + do, +.09)C], 


which we recognise as the MGF for a Gaussian with mean p; + fy and variance 
a; +63. Thus, Z is also Gaussian distributed: Z ~ N( + po, 07 +03). 

A similar calculation may be performed to calculate the PDF of the random 
variable W = X — Y. If we introduce the variable Y = —Y then W = X+Y, 
where Y ~ N(—ju, 7). Thus, using the result above, we find W ~ N(u1 — 
lb, ot + 03). 


> An executive travels home from her office every evening. Her journey consists of a train 
ride, followed by a bicycle ride. The time spent on the train is Gaussian distributed with 
mean 52 minutes and standard deviation 1.8 minutes, while the time for the bicycle journey 
is Gaussian distributed with mean 8 minutes and standard deviation 2.6 minutes. Assuming 
these two factors are independent, estimate the percentage of occasions on which the whole 
journey takes more than 65 minutes. 


We first define the random variables 
X = time spent on train, Y = time spent on bicycle, 


so that X ~ N(52,(1.8)?) and Y ~ N(8,(2.6)). Since X and Y are independent, the total 
journey time T = X + Y is distributed as 


T ~ N(52+8, (1.8)° + (2.6)”) = N(60, (3.16)’). 


The standard variable is thus 


and the required probability is given by 
65 — 60 
3.16 


Thus the total journey time exceeds 65 minutes on 5.7% of occasions. < 


Pr(T > 65) = Pr (z > ) = Pr(Z > 1.58) = 1 — 0.943 = 0.057. 
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The above results may be extended. For example, if the random variables 
X;, i = 1,2,...,n, are distributed as X; ~ N(ti, 07) then the random variable 
Z = >; c:X; (where the c; are constants) is distributed as Z ~ N(°; citi, 90; ¢?.07). 


30.9.2 The log-normal distribution 


If the random variable X follows a Gaussian distribution then the variable 
Y = e* is described by a log-normal distribution. Clearly, if X can take values 
in the range —oo to oo, then Y will lie between 0 and oo. The probability density 
function for Y is found using the result (30.58). It is 


dx 1 1 | eee 

—| = —=- exp |-— >]. 

dy o./2ny 20 

We note that yw and o? are not the mean and variance of the log-normal 
distribution, but rather the parameters of the corresponding Gaussian distribution 
for X. The mean and variance of Y, however, can be found straightforwardly 
using the MGF of X, which reads My(t) = E[e'*] = exp(ut 4 50717). Thus, the 
mean of Y is given by 


g(y) = f(xy) 


E[Y] = Ele“] = My(1) =exp(u + 407), 


L 
2 


and the variance of Y reads 
VY] = E[Y7] —(E[Y])? = Ele*] — (Ele*]y 
= Mx(2) — [Mx(1)]? = exp(2u + 0”)[exp(”) — 1]. 


In figure 30.15, we plot some examples of the log-normal distribution for various 
values of the parameters p and o?. 


30.9.3 The exponential and gamma distributions 


The exponential distribution with positive parameter / is given by 


a de® for x > 0, (30.116) 
x)= 
0 for x <0 


and satisfies fine f(x) dx = 1 as required. The exponential distribution occurs nat- 
urally if we consider the distribution of the length of intervals between successive 
events in a Poisson process or, equivalently, the distribution of the interval (i.e. 
the waiting time) before the first event. If the average number of events per unit 
interval is J then on average there are Ax events in interval x, so that from the 
Poisson distribution the probability that there will be no events in this interval is 
given by 
Ax 


Pr(no events in interval x) =e. 
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Figure 30.15 The PDF g(y) for the log-normal distribution for various values 
of the parameters y and o. 


The probability that an event occurs in the next infinitestimal interval [x, x + dx] 
is given by Adx, so that 


Pr(the first event occurs in interval [x,x + dx]) = e**Adx. 
Hence the required probability density function is given by 
f(x) =e. 
The expectation and variance of the exponential distribution can be evaluated as 
1/4 and (1/4)? respectively. The MGF is given by 
2 

At 
We may generalise the above discussion to obtain the PDF for the interval 


between every rth event in a Poisson process or, equivalently, the interval (waiting 
time) before the rth event. We begin by using the Poisson distribution to give 


M(t) = (30.117) 


(4 r—1 
Pr(r — 1 events occur in interval x) = ex Vx 
(r—1)! 
from which we obtain 
Fle r—1 
Pr(rth event occurs in the interval [x, x + dx]) =e“ ‘ x) nee 
Thus the required PDF is 
{a= 7 _ 7 (ax) 1e**, (30.118) 


which is known as the gamma distribution of order r with parameter 4. Although 
our derivation applies only when r is a positive integer, the gamma distribution is 
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Figure 30.16 The PDF f(x) for the gamma distributions y(A,r) with 2 = 1 
and r = 1,2,5,10. 


defined for all positive r by replacing (r—1)! by I(r) in (30.118); see the appendix 
for a discussion of the gamma function I(x). If a random variable X is described 
by a gamma distribution of order r with parameter J, we write X ~ y(A,r); 
we note that the exponential distribution is the special case y(A, 1). The gamma 
distribution (A,r) is plotted in figure 30.16 for 4 = 1 and r = 1,2,5,10. For 
large r, the gamma distribution tends to the Gaussian distribution whose mean 
and variance are specified by (30.120) below. 

The MGF for the gamma distribution is obtained from that for the exponential 
distribution, by noting that we may consider the interval between every rth event 
in a Poisson process as the sum of r intervals between successive events. Thus the 
rth-order gamma variate is the sum of r independent exponentially distributed 
random variables. From (30.117) and (30.90), the MGF of the gamma distribution 
is therefore given by 

A-t 


M(t) = (4) ; (30.119) 
from which the mean and variance are found to be 
r r 
E[X] =>, VIX] = 5. (30.120) 


We may also use the above MGF to prove another useful theorem regarding 
multiple gamma distributions. If X; ~ y(/,7r;), i = 1,2,...,n, are independent 
gamma variates then the random variable Y = X; + X.+---+ X, has MGF 


n 4 rj XZ retry thn 
mw =[T (=) = (4) . (30.121) 


i=1 


Thus Y is also a gamma variate, distributed as Y ~ p(/,ry +ro +--+ +1,)- 
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30.9.4 The chi-squared distribution 


In subsection 30.6.2, we showed that if X is Gaussian distributed with mean yu and 
variance a7, such that X ~ N(u,07), then the random variable Y = (x — p)*/o? 
is distributed as the gamma distribution Y ~ »(4, 4). Let us now consider n 
independent Gaussian random variables X; ~ N(ui, 07), i=1,2,...,n, and define 
the new variable 


n 
Xi — mi) 
£z=>. coe (30.122) 
i=1 i 


Using the result (30.121) for multiple gamma distributions, 72 must be distributed 
as the gamma variate 7? ~ 5. 5), which from (30.118) has the PDF 


1 


tta= rl ten 2-1 exp(—1y?) 
5 


(2)! exp(— ty). (30.123) 


~ 29?2T(En) 
This is known as the chi-squared distribution of order n and has numerous 
applications in statistics (see chapter 31). Setting 1 = 5 and r = 5n in (30.120), 
we find that 


E[g]=n, V[e] = 2h. 


An important generalisation occurs when the n Gaussian variables X; are not 
linearly independent but are instead required to satisfy a linear constraint of the 
form 


cyX, +o2Xo +++ + e,X, = 0, (30.124) 


in which the constants c; are not all zero. In this case, it may be shown (see 
exercise 30.40) that the variable y7 defined in (30.122) is still described by a chi- 
squared distribution, but one of order n — 1. Indeed, this result may be trivially 
extended to show that if the n Gaussian variables X; satisfy m linear constraints 
of the form (30.124) then the variable y2 defined in (30.122) is described by a 
chi-squared distribution of order n — m. 


30.9.5 The Cauchy and Breit-Wigner distributions 


A random variable X (in the range —oo to 00) that obeys the Cauchy distribution 
is described by the PDF 


fy=+— 


nm 1i+x?° 
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Figure 30.17 The PDF f(x) for the Breit-Wigner distribution for different 
values of the parameters xo and I. 


This is a special case of the Breit-Wigner distribution 


1 
f= wooo 
ql? + (x — X0) 
which is encountered in the study of nuclear and particle physics. In figure 30.17, 
we plot some examples of the Breit—Wigner distribution for several values of the 
parameters xp and I. 

We see from the figure that the peak (or mode) of the distribution occurs 
at x = xo. It is also straightforward to show that the parameter I’ is equal to 
the width of the peak at half the maximum height. Although the Breit—-Wigner 
distribution is symmetric about its peak, it does not formally possess a mean since 
the integrals fe xf(x)dx and Aes xf(x)dx both diverge. Similar divergences occur 
for all higher moments of the distribution. 


30.9.6 The uniform distribution 


Finally we mention the very simple, but common, uniform distribution, which 
describes a continuous random variable that has a constant PDF over its allowed 
range of values. If the limits on X are a and b then 


: 1/(b—a) fora<x<b, 
f(x) = . 
0 otherwise. 


The MGF of the uniform distribution is found to be 
ebt — elt 
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and its mean and variance are given by 


b b—ay? 
ey= vee! =e 


30.10 The central limit theorem 


In subsection 30.9.1 we discussed approximating the binomial and Poisson distri- 
butions by the Gaussian distribution when the number of trials is large. We now 
discuss why the Gaussian distribution is so common and therefore so important. 
The central limit theorem may be stated as follows. 


Central limit theorem. Suppose that X;, i = 1,2,...,n, are independent random 
variables, each of which is described by a probability density function f(x) (these 
may all be different) with a mean ju; and a variance o?. The random variable Z = 
(0; Xi) /n, ie. the ‘mean’ of the X;, has the following properties: 

(i) its expectation value is given by E[Z] = (30, i) /n: 

(ii) its variance is given by V[Z] = (30; 4?) /n’: 

(ili) as n — © the probability function of Z tends to a Gaussian with corre- 

sponding mean and variance. 


We note that for the theorem to hold, the probability density functions f;(x) 
must possess formal means and variances. Thus, for example, if any of the X; 
were described by a Cauchy distribution then the theorem would not apply. 

Properties (i) and (1i) of the theorem are easily proved, as follows. Firstly 


Si Hi 
7? 


E(Z] = S(E[Xi] + ELK) +--+ LX) = (un to bot te) = 


a result which does not require that the X; are independent random variables. If 
Hi; = pw for all i then this becomes 


ny 
E[Z]=—-=u 


Secondly, if the X; are independent, it follows from an obvious extension of 
(30.68) that 


ll 


1 
V[Z]=V Pace + Xo+---+X) 


2 
4 (V[Xi] + V[Xo] + --- + VEXn)) = Ligh 
is n 


ll 


Let us now consider property (iii), which is the reason for the ubiquity of 
the Gaussian distribution and is most easily proved by considering the moment 
generating function Mz(t) of Z. From (30.90), this MGF is given by 


Mz(t) = [[ (5) ; 
i=1 
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where Mx,(t) is the MGF of fj(x). Now 


t t t 
Mx. ( ) a+ E[Xj] +3 5E[X7]+-:- 
n n n 


t t 
Hl me tap +uate 


and as n becomes large 


t t 
ats (5) xen (i+ tet), 


as may be verified by expanding the exponential up to terms including (t/n). 
Therefore 


n 2 
it t 
mein = Tex (4 +4095) = exp (2 Hi 4 pa e), 
i=l 


Comparing this with the form of the MGF for a Gaussian distribution, (30.114), 
we can see that the probability density function g(z) of Z tends to a Gaussian dis- 
tribution with mean 5°; y;/n and variance >; o;/n?. In particular, if we consider 
Z to be the mean of n independent measurements of the same random variable X 
(so that X; = X fori =1,2,...,n) then, as n > 0, Z has a Gaussian distribution 
with mean p and variance o?/n. 

We may use the central limit theorem to derive an analogous result to (iii) 
above for the product W = X,X2---X, of the n independent random variables 
X;. Provided the X; only take values between zero and infinity, we may write 


InW =1nX, +1nX).+---+1nX), 


which is simply the sum of n new random variables In X;. Thus, provided these 
new variables each possess a formal mean and variance, the PDF of InW will 
tend to a Gaussian in the limit n — 00, and so the product W will be described 
by a log-normal distribution (see subsection 30.9.2). 


30.11 Joint distributions 


As mentioned briefly in subsection 30.4.3, it is common in the physical sciences to 
consider simultaneously two or more random variables that are not independent, 
in general, and are thus described by joint probability density functions. We will 
return to the subject of the interdependence of random variables after first 
presenting some of the general ways of characterising joint distributions. We 
will concentrate mainly on bivariate distributions, 1. distributions of only two 
random variables, though the results may be extended readily to multivariate 
distributions. The subject of multivariate distributions is large and a detailed 
study is beyond the scope of this book; the interested reader should therefore 
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consult one of the many specialised texts. However, we do discuss the multinomial 
and multivariate Gaussian distributions, in section 30.15. 

The first thing to note when dealing with bivariate distributions is that the 
distinction between discrete and continuous distributions may not be as clear as 
for the single variable case; the random variables can both be discrete, or both 
continuous, or one discrete and the other continuous. In general, for the random 
variables X and Y, the joint distribution will take an infinite number of values 
unless both X and Y have only a finite number of values. In this chapter we 
will consider only the cases where X and Y are either both discrete or both 
continuous random variables. 


30.11.1 Discrete bivariate distributions 


In direct analogy with the one-variable (univariate) case, if X is a discrete random 
variable that takes the values {x;} and Y one that takes the values {y,} then the 
probability function of the joint distribution is defined as 


: Pr(X =x; Y =y;) forx=x, y=y;, 
f(xy) = : Rs ; 
0 otherwise. 


We may therefore think of f(x, y) as a set of spikes at valid points in the xy-plane, 
whose height at (xj, yi) represents the probability of obtaining X = x; and Y = yj. 
The normalisation of f(x, y) implies 


SS fey) =1, (30.125) 
ij 


where the sums over i and j take all valid pairs of values. We can also define the 
cumulative probability function 


F(x y) = >> >> fOay;), (30.126) 


XiSX YjSV 


from which it follows that the probability that X lies in the range [a), a7] and Y 
lies in the range [b;, bo] is given by 


Pr(a; < X < ap, by < Y < by) = F(a, bz) — F(a, bz) — F(a, b1) + F(ay, 51). 


Finally, we define X and Y to be independent if we can write their joint distribution 
in the form 


f(xy) = fxO)fy(y), (30.127) 


ie. as the product of two univariate distributions. 
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30.11.2 Continuous bivariate distributions 


In the case where both X and Y are continuous random variables, the PDF of 
the joint distribution is defined by 


f(x,y) dx dy = Pr(x < X <x+dx, y< Y < y+ dy), 
(30.128) 


so f(x,y) dx dy is the probability that x lies in the range [x,x + dx] and y lies in 
the range [y, y + dy]. It is clear that the two-dimensional function f(x, y) must be 
everywhere non-negative and that normalisation requires 


fe fx y)dxdy =1. 
It follows further that 


by 2 
Pr(ay < X < a, db) < Y < bo) -| / f(x, y) dx dy. 
Bin 4 (30.129) 


We can also define the cumulative probability function by 
F(x,y) =Pr(X¥ <x, Y<y)= in [ f(u,v) dudv, 
from which we see that (as for the discrete ered 
Pr(ay < X <a, by) < Y < by) = F(a), bo) — F(a1, bo) — F(do, bi) + F(a, 51). 


Finally we note that the definition of independence (30.127) for discrete bivariate 
distributions also applies to continuous bivariate distributions. 


>A flat table is ruled with parallel straight lines a distance D apart, and a thin needle of 
length | < D is tossed onto the table at random. What is the probability that the needle 
will cross a line? 


Let 0 be the angle that the needle makes with the lines, and let x be the distance from 
the centre of the needle to the nearest line. Since the needle is tossed ‘at random’ onto 
the table, the angle 0 is uniformly distributed in the interval [0,2], and the distance x 
is uniformly distributed in the interval [0,D/2]. Assuming that 0 and x are independent, 
their joint distribution is just the product of their individual distributions, and is given by 
1 1 2 
0,x) = -—~ = —. 
F(@x) nD/2 xD 
The needle will cross a line if the distance x of its centre from that line is less than 31 sin 0. 
Thus the required probability is 


oe 1 
21 ‘ 21 
=f I w= => | sin dé = 


This gives an experimental (but cumbersome) method of determining z. < 
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30.11.3: Marginal and conditional distributions 


Given a bivariate distribution f(x,y), we may be interested only in the proba- 
bility function for X irrespective of the value of Y (or vice versa). This marginal 
distribution of X is obtained by summing or integrating, as appropriate, the 
joint probability distribution over all allowed values of Y. Thus, the marginal 
distribution of X (for example) is given by 


fr) = ie f(x, yj) for a discrete distribution, (30.130) 


{ f(x. y)dy for a continuous distribution. 


It is clear that an analogous definition exists for the marginal distribution of Y. 

Alternatively, one might be interested in the probability function of X given 
that Y takes some specific value of Y = yo, 1.e. Pr(X = x|Y = yo). This conditional 
distribution of X is given by 


F(X, Yo) 

fy (vo) 

where fy(y) is the marginal distribution of Y. The division by fy(yo) is necessary 
in order that g(x) is properly normalised. 


g(x) = 


30.12 Properties of joint distributions 


The probability density function f(x, y) contains all the information on the joint 
probability distribution of two random variables X and Y. In a similar manner 
to that presented for univariate distributions, however, it is conventional to 
characterise f(x,y) by certain of its properties, which we now discuss. Once 
again, most of these properties are based on the concept of expectation values, 
which are defined for joint distributions in an analogous way to those for single- 
variable distributions (30.46). Thus, the expectation value of any function g(X, Y) 
of the random variables X and Y is given by 


Di 80%, VAS (Xi, Y/) for the discrete case, 


Ele(X,Y)] = 8 ii 
le( M fe f°, 80. y)f (x, y)dx dy for the continuous case. 


30.12.1 Means 


The means of X and Y are defined respectively as the expectation values of the 
variables X and Y . Thus, the mean of X is given by 


ei dy Xf (1, Ys) for the discrete case, 


Bed = me =| 20 . Ix dy for th ; 
if. fe xf(x,y)dxdy for the continuous case. (30.131) 


E[Y] is obtained in a similar manner. 
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Show that if X and Y are independent random variables then E[X Y] = E[X]E[Y]. 


Let us consider the case where X and Y are continuous random variables. Since X and 
Y are independent f(x, y) = fx(x)fy(y), so that 


eixy|= [0 | xfeoofyyyasdy= [ xfetoyax [ yfvyydy = ELIELY). 


An analogous proof exists for the discrete case. < 


30.12.2 Variances 


The definitions of the variances of X and Y are analogous to those for the 
single-variable case (30.48), i.e. the variance of X is given by 
> re ij (xi — Ex)? f(x, Y)) for the discrete case, 


V[X]) =o, = 
] 4 Jo, [E.(& — ux)? f(x,y) dx dy for the continuous case. (30.132) 


Equivalent definitions exist for the variance of Y. 


30.12.3 Covariance and correlation 


Means and variances of joint distributions provide useful information about 
their marginal distributions, but we have not yet given any indication of how to 
measure the relationship between the two random variables. Of course, it may 
be that the two random variables are independent, but often this is not so. For 
example, if we measure the heights and weights of a sample of people we would 
not be surprised to find a tendency for tall people to be heavier than short people 
and vice versa. We will show in this section that two functions, the covariance 
and the correlation, can be defined for a bivariate distribution and that these are 
useful in characterising the relationship between the two random variables. 
The covariance of two random variables X and Y is defined by 


Cov[X, Y] = E[(X — ux)(Y — py)]), (30.133) 


where px and py are the expectation values of X and Y respectively. Clearly 
related to the covariance is the correlation of the two random variables, defined 
by 

Corr[X, Y] = Sens M1. (30.134) 

OxOy 

where oy and oy are the standard deviations of X and Y respectively. It can be 
shown that the correlation function lies between —1 and +1. If the value assumed 
is negative, X and Y are said to be negatively correlated, if it is positive they are 
said to be positively correlated and if it is zero they are said to be uncorrelated. 
We will now justify the use of these terms. 
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One particularly useful consequence of its definition is that the covariance 
of two independent variables, X and Y, is zero. It immediately follows from 
(30.134) that their correlation is also zero, and this justifies the use of the term 
‘uncorrelated’ for two such variables. To show this extremely important property 
we first note that 


Cov[X, Y] = E[(X — ux)(Y — py)] 
= E[XY —puxY —pyX + uxpy] 
= E[XY] — wyE[Y] — wy E[X] + exuy 
= FIXY] — uses (30.135) 


Now, if X and Y are independent then E[XY] = E[X]E[Y] = uxpy and so 
Cov[X, Y] = 0. It is important to note that the converse of this result is not 
necessarily true; two variables dependent on each other can still be uncorrelated. 
In other words, it is possible (and not uncommon) for two variables X and Y 
to be described by a joint distribution f(x,y) that cannot be factorised into a 
product of the form g(x)h(y), but for which Corr[X, Y] = 0. Indeed, from the 
definition (30.133), we see that for any joint distribution f(x, y) that is symmetric 
in x about fy (or similarly in y) we have Corr[X, Y] = 0. 

We have already asserted that if the correlation of two random variables is 
positive (negative) they are said to be positively (negatively) correlated. We have 
also stated that the correlation lies between —1 and +1. The terminology suggests 
that if the two RVs are identical (ie. ¥ = Y) then they are completely correlated 
and that their correlation should be +1. Likewise, if ¥ = —Y then the functions 
are completely anticorrelated and their correlation should be —1. Values of the 
correlation function between these extremes show the existence of some degree 
of correlation. In fact it is not necessary that X = Y for Corr[X, Y] = 1; it is 
sufficient that Y is a linear function of X, ie. Y = aX +b (with a positive). If a 
is negative then Corr[X, Y] = —1. To show this we first note that wy = apy +b. 
Now 


Y =aX +b =aX + ply — apy Y —py =a(X — py), 
and so using the definition of the covariance (30.133) 
Cov[X, Y] = aE[(X — px)*] = aay. 


It follows from the properties of the variance (subsection 30.5.3) that oy = |alax 
and so, using the definition (30.134) of the correlation, 


9) 
ao a 
Corr[X, Y] = * =—, 
lalox al 


which is the stated result. 
It should be noted that, even if the possibilities of X¥ and Y being non-zero are 
mutually exclusive, Corr[X, Y] need not have value +1. 
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>A biased die gives probabilities SP, P,P, P, p, 2p of throwing 1, 2, 3, 4, 5, 6 respectively. 
If the random variable X is the number shown on the die and the random variable Y is 
defined as X*, calculate the covariance and correlation of X and Y. 


We have already calculated in subsections 30.2.1 and 30.5.4 that 


253 480 


2 53 5 
| one) E[X] = 73> E(x] => = T69" 


13 
Using (30.135), we obtain 
Cov[X, Y] = Cov[X, X7] = E[X?] — E[X]E[X?]. 
Now E[X°] is given by 
E(X*] = x 4p + (B4+HR%+4+445°)p +6 x 2p 


1313 
a =>? = 101, 


and the covariance of X and Y is given by 


53 253 = 3660 


Cov[X, ¥] = 101-5 x = = Te 


The correlation is defined by Corr[X, Y] = Cov[X, Y]/oxyoy. The standard deviation of 


Y may be calculated from the definition of the variance. Letting wy = E[X?] = oe gives 


oF = 5 (py) +p (2 my) +p (8 —ay) +P (#— ay)” 
i 
T 


p(s =e) op (6 Sie) 
_ 187356 28.824 


169? 169 ~ 


We deduce that 


3660 | 169 [169 
Corr[X, Y] = ae ae = 0.984. 


Thus the random variables X and Y display a strong degree of positive correlation, as we 
would expect. < 


We note that the covariance of X and Y occurs in various expressions. For 
example, if X and Y are not independent then 


VIX+Y]=E[(X+Y)] -(E[x + ¥])? 
= E [X?] + 2E[XY]+ E [Y’] — {(E[X]) + 2E[X]E[Y] + (E[Y)} 
= V[X] + V[Y] + 2(E[XY] — E[X]E[Y]) 
= V[X] + V[Y] +2 Cov[X, Y]. 
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More generally, we find (for a, b and c constant) 


V[aX + bY +c] =a°V[X] + B°V[Y] + 2ab Cov[X, Y]. 
(30.136) 


Note that if X and Y are in fact independent then Cov[X, Y] = 0 and we recover 
the expression (30.68) in subsection 30.6.4. 

We may use (30.136) to obtain an approximate expression for V[f(X, Y)] 
for any arbitrary function f, even when the random variables X and Y are 
correlated. Approximating f(X,Y) by the linear terms of its Taylor expansion 
about the point (1x, uy), we have 


S(X,Y) © flux, uy) + (4) (X — x) + (sr) (Y — ny), 


where the partial derivatives are evaluated at X = wy and Y = py. Taking the 
variance of both sides, and using (30.136), we find 


OL OEY’ ies of \ ( of 
VUS(X,Y)] = (=) V[X] + (sr) viv +2 (52) (+) elo 


Clearly, if Cov[X, Y] = 0, we recover the result (30.69) derived in subsection 30.6.4. 
We note that (30.138) is exact if f(X, Y) is linear in X and Y. 

For several variables Xj, i = 1,2,...,n, we can define the symmetric (positive 
definite) covariance matrix whose elements are 


Vij = Cov[Xi, Xj], (30.139) 
and the symmetric (positive definite) correlation matrix 

pij = Corr[X;, Xj]. 
The diagonal elements of the covariance matrix are the variances of the variables, 


whilst those of the correlation matrix are unity. For several variables, (30.138) 
generalises to 


ened ne an 
VIP XM * DO (SL) vEl+ OY (SE) (FE) commu 
F i 1 J 


i j#i 


where the partial derivatives are evaluated at X; = uy,. 
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>A card is drawn at random from a normal 52-card pack and its identity noted. The card 
is replaced, the pack shuffled and the process repeated. Random variables W, X, Y, Z are 
defined as follows: 


W= if the drawn card is a heart; W = 0 otherwise. 

X =4 _ if the drawn card is an ace, king, or queen; X = 2 if the card is 

a jack or ten; X = 0 otherwise. 

Y =1_ if the drawn card is red; Y =0 otherwise. 

Z=2__ if the drawn card is black and an ace, king or queen; Z =0 
otherwise. 


Establish the correlation matrix for W, X, Y, Z. 


bw =2X 455 w= (4% 3) +(2* 8) = 3 


The variances, calculated from oj, = V[U] = E [U7] —(E[U])’, where U = W, X, Y or 
Z, are 


2 
(13) = Tes> 


2 
ow = (4% 4)— (3) = 9% = (16x 3) + (4 #) 
2 2 
oy = (1x 3) — (3) 07 = (4% 5) — (3) = too 
The covariances are found by first calculating E[W X] etc. and then forming E[W X]—uw py 
etc. 


E[WX] = 2(4)(3) +2(2)(4)=8, Covlw,x] = 4-4 (4) =0, 
E[WY] =2(1) (4) =3, Cov[W,Y] =4—1(4) =1, 
E([WZ] =0, Cov[W,Z] =0-—43(4) =-#. 
E[XY] =4()($) +2 ($)=8. Covix,y] = 8-#(4) =0, 
E[XZ] = 4(2)($) = 4: Cov[X,Z] = 28 — 46 (3) = 3%, 
E[YZ] =0, Cov[Y,Z] =0—4(4) =-#. 


The correlations Corr[W, X] and Corr[X, Y] are clearly zero; the remainder are given by 


Corr[W,Y] =4 (3x4)? = 0.577, 


Corr[W,Z] =—3 (3x &)"” = -0.209, 


Corr[X,Z] = #8 (4B x 8)” = 0.598, 


To) * 169 


Corr[Y,Z] =—2 (1 x 8)? = -0.361. 


Finally, then, we can write down the correlation matrix: 


1 0 058 —0.21 
- 0 1 0 0.60 
P= 058 0 1 0.36 


30.13 GENERATING FUNCTIONS FOR JOINT DISTRIBUTIONS 


As would be expected, X is uncorrelated with either W or Y, colour and face-value being 
two independent characteristics. Positive correlations are to be expected between W and 
Y and between X and Z; both correlations are fairly strong. Moderate anticorrelations 
exist between Z and both W and Y, reflecting the fact that it is impossible for W and Y 
to be positive if Z is positive. < 


Finally, let us suppose that the random variables X;, i = 1,2,...,n, are related 
to a second set of random variables Y, = Y;(X1, X2,...,Xn), k = 1,2,...,m. By 
expanding each Y; as a Taylor series as in (30.137) and inserting the resulting 
expressions into the definition of the covariance (30.133), we find that the elements 
of the covariance matrix for the Y; variables are given by 


oY, oY, 
Cov[Y¥;, Yi] * es ($) (=) Cov[X;, Xj]. 
Poof } J 


It is straightforward to show that this relation is exact if the Y, are linear 
combinations of the X;. Equation (30.140) can then be written in matrix form as 


(30.140) 


Vy =SVxS!, (30.141) 


where Vy and Vy are the covariance matrices of the Y, and X; variables respec- 
tively and S is the rectangular m x n matrix with elements S;; = 0Y,/0X;. 


30.13 Generating functions for joint distributions 


It is straightforward to generalise the discussion of generating function in section 
30.7 to joint distributions. For a multivariate distribution f(X1, X2,...,Xn) of 
non-negative integer random variables Xj, i = 1,2,...,n, we define the probability 
generating function to be 


X,.X. Xs 
M(t), t2,...,tn) = Elty'ts? +--+ t,"]. 


n 


As in the single-variable case, we may also define the closely related moment 
generating function, which has wider applicability since it is not restricted to 
non-negative integer random variables but can be used with any set of discrete 
or continuous random variables X; (i = 1,2,...,n). The MGF of the multivariate 
distribution f(X1, X2,...,X,) is defined as 


M(t1,to,...,tn) = Efe'*! e2*2... elnXn] _ E [ett Xt eXot tin 
(30.142) 


and may be used to evaluate (joint) moments of f(X1, X2,...,Xn). By performing 
a derivation analogous to that presented for the single-variable case in subsection 
30.7.2, it can be shown that 

omitmat+-+im M (0, 0, ee, 0) 


1 
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Finally we note that, by analogy with the single-variable case, the characteristic 
function and the cumulant generating function of a multivariate distribution are 
defined respectively as 


C(ti, t2,.-.5tn) = M(iti,ita,...,itn) and = K(t1, f2,...,tn) = In M(t, f2,..., tn). 


> Suppose that the random variables X;, i= 1,2,...,n, are described by the PDF 
f(x) = f (x1, X25... Xn) = N exp(—$x7 Ax), 


where the column vector x =(x, Xz +++ Xn»)', A is ann Xn symmetric matrix and N 
is a normalisation constant such that 


[ toorxs ff ff TOD do ee 


Find the MGF of f(x). 


From (30.142), the MGF is given by 

M(t, t2,...5tn) = nf exp(—4x" Ax + tx) d"x, (30.144) 
where the column vectort=(t; ft ++: t,)'. In order to evaluate this multiple integral, 
we begin by noting that 
xTAx — 2tx = (x — AT! t)TA(x — Alt) — tT Att 


which is the matrix equivalent of ‘completing the square’. Using this expression in (30.144) 
and making the substitution y = x — A7't, we obtain 


M(t, to,..-5tn) = cexp(5t™A7't), (30.145) 


where the constant c is given by 
c= vf exp(—4y" Ay) d’y. 


From the normalisation condition for N, we see that c = 1, as indeed it must be in order 
that M(0,0,...,0) = 1. < 


30.14 Transformation of variables in joint distributions 


Suppose the random variables X;, i = 1,2,...,n, are described by the multivariate 
PDF f(x1,X2...,X;,). If we wish to consider random variables Y;, j = 1,2,...,m, 
related to the X; by Y; = Yj(X1, X2,..., Xm) then we may calculate g(y1, y2,..-, Vm) 
the PDF for the Y;, in a similar way to that in the univariate case by demanding 
that 


|f(X1,X2...,Xn) dx dx2 +++ dXn| = |g(V1,V25---s Ym) dy1 dy2 +++ dyn. 


From the discussion of changing the variables in multiple integrals given in 
chapter 6 it follows that, in the special case where n = m, 


g(V1, Y25- os »Vm) = f(X1, X2. on XDI; 
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where 
Ox, OXn 
By oe 
_ O(X1,X2--25Xn) : ie 
ame’, an . : . 
(V1, ¥25+-+5 Yn) By: Ox, 
OVn = OYn 


is the Jacobian of the x; with respect to the yj. 


> Suppose that the random variables X;, i = 1,2,...,n, are independent and Gaussian dis- 
tributed with means jy; and variances a? respectively. Find the PDF for the new variables 
Zi = (Xi — wi)/o;, i = 1,2,...,n. By considering an elemental spherical shell in Z-space, 
find the PDF of the chi-squared random variable 43 = > j_, Z?. 


Since the X; are independent random variables, 


1 n a : 2 
Fleer 25-050) = Soer)f (2) Fn) = exp ew | 
n i=1 i 


(22)"/2G107+++6 
To derive the PDF for the variables Z;, we require 
IP (X1, X25-++, Xn) dX1 AX2 +++ AXn| = |8(Z1,22,--+5 Zn) d21 d22 +++ d2nl, 


and, noting that dz; = dx;/o;, we obtain 


1 1 n 
&(21,22,-+-52n) = Gaya oP (-3 ¥ =) : 


Let us now consider the random variable 47 = 7j_, Z?, which we may regard as the 
square of the distance from the origin in the n-dimensional Z-space. We now require that 
8 (21, 225-++52n) dz1 za ++» dn = Wg) hn 
If we consider the infinitesimal volume dV = dz, dz,---dz, to be that enclosed by the 
n-dimensional spherical shell of radius x, and thickness dy, then we may write dV = 

Ax dyn, for some constant A. We thus obtain 
Wyn d%n © exP(—F%n)%n din exP(—FLn)xn Ans 


where we have used the fact that dy? = 2y,, dz,. Thus we see that the PDF for 2 is given 
by 
hy) = Bexp(—3%n)zn 
for some constant B. This constant may be determined from the normalisation condition 
[andi 1 
0 


and is found to be B = [2"?I(4n)]~'. This is the nth-order chi-squared distribution 
discussed in subsection 30.9.4. < 


30.15 Important joint distributions 


In this section we will examine two important multivariate distributions, the 
multinomial distribution, which is an extension of the binomial distribution, and 
the multivariate Gaussian distribution. 


1207 


PROBABILITY 


30.15.1 The multinomial distribution 


The binomial distribution describes the probability of obtaining x ‘successes’ from 
n independent trials, where each trial has only two possible outcomes. This may 
be generalised to the case where each trial has k possible outcomes with respective 
probabilities p;, po, ..., py. If we consider the random variables X;, i = 1,2,...,n, 
to be the number of outcomes of type i in n trials then we may calculate their 
joint probability function 


f(X1,X2,...,Xk) = Pr(X1 = x1, X2 = x2, ..., Xk = Xx), 


where we must have oar? =n. In n trials the probability of obtaining x; 
outcomes of type 1, followed by x2 outcomes of type 2 etc. is given by 


Pi py oY? a Pi" 
However, the number of distinguishable permutations of this result is 
n! 


> 


Xp lxo!---x,! 
and thus 


n! 


f (1, X2,...,X4) = ecu ee ase Pi. (30.146) 


This is the multinomial probability distribution. 

If k = 2 then the multinomial distribution reduces to the familiar binomial 
distribution. Although in this form the binomial distribution appears to be a 
function of two random variables, it must be remembered that, in fact, since 
p2 =1—p,; and x, =n—x,, the distribution of X, is entirely determined by the 
parameters p and n. That X, has a binomial distribution is shown by remembering 
that it represents the number of objects of a particular type obtained from 
sampling with replacement, which led to the original definition of the binomial 
distribution. In fact, any of the random variables X; has a binomial distribution, 
ie. the marginal distribution of each X; is binomial with parameters n and pj. It 
immediately follows that 


E(X]=np; and V[Xi]* = npi(1— pi). (30.147) 
> At a village féte patrons were invited, for a 10p entry fee, to pick without looking six 


tickets from a drum containing equal large numbers of red, blue and green tickets. If five 
or more of the tickets were of the same colour a prize of 100 p was awarded. A consolation 


award of 40 p was made if two tickets of each colour were picked. Was a good time had by 
all? 


In this case, all types of outcome (red, blue and green) have the same probabilities. The 
probability of obtaining any given combination of tickets is given by the multinomial 
distribution with n = 6, k = 3 and p; = 4, i= 1,2,3. 


3 
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(i) The probability of picking six tickets of the same colour is given by 


Pr (six of the same colour) = 3 x a eset : 
6!0!0! \3 3 3 243° 


The factor of 3 is present because there are three different colours. 
(ii) The probability of picking five tickets of one colour and one ticket of another 
colour is 


Pr(five of one colour; one of another) = 3 x 2 x oi Lye 1)" (1 oe 4 
: i 51110! \3 3 3) 81 


The factors of 3 and 2 are included because there are three ways to choose the 
colour of the five matching tickets, and then two ways to choose the colour of the 
remaining ticket. 

(iii) Finally, the probability of picking two tickets of each colour is 


' 2 2 2 
Pr (two of each colour) = — (3) (3) (3) = - 


Thus the expected return to any patron was, in pence, 


1 4 10 
100 (a + =) + (40 x x) = 10.29. 


A good time was had by all but the stallholder! < 


30.15.2 The multivariate Gaussian distribution 


A particularly interesting multivariate distribution is provided by the generalisa- 
tion of the Gaussian distribution to multiple random variables Xj, i = 1,2,...,n. 
If the expectation value of X; is E(X;) = u; then the general form of the PDF is 
given by 


f (%1,X2,...,X%n) = N exp -4 SoS ay: — Hi )(xj — Hy), 
ij 


where aj; = aj; and N is a normalisation constant that we give below. If we write 
the column vectors x = (xj X2 -°:: Xn)’ and w= (mu fo +*: Mn)’, and 
denote the matrix with elements aj; by A then 


F(x) = f(X1,X2,--+5 Xn) = Nexp [—3(x — 1) AK— 1], 


where A is symmetric. Using the same method as that used to derive (30.145) it 
is straightforward to show that the MGF of f(x) is given by 


M(t, ty,....t) = exp (ut + St™A't), 


where the column matrix t=(t; ft: ---  t,)'. From the MGF, we find that 
0?M(0,0,...,0) = 
E[X;:Xj] = ata, = pip; + (A")iy, 
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and thus, using (30.135), we obtain 


Cov[X;, Xj] = E[(Xi — u;)(Xj — wy) = (A). 
Hence A is equal to the inverse of the covariance matrix V of the Xj, see (30.139). 
Thus, with the correct normalisation, f(x) is given by 


f%) 2 


~ (my(det v2 *P [—$(«— w)'V1(x— p)]- 


(30.148) 


> Evaluate the integral 


I fol 5 w)] a"x, 


where V is a symmetric matrix, and hence verify the normalisation in (30.148). 


We begin by making the substitution y = x — u to obtain 


I -| exp(—4y"V~ly) d’y. 


va) 


Since V is a symmetric matrix, it may be diagonalised by an orthogonal transformation to 
the new set of variables y’ = S'y, where S is the orthogonal matrix with the normalised 
eigenvectors of V as its columns (see section 8.16). In this new basis, the matrix V becomes 


V=S'VS = diag(A1,42,...,4n), 
where the /; are the eigenvalues of V. Also, since S is orthogonal, det S = +1, and so 
d"y = |det S|d"y' =d"y’. 


Thus we can write I as 


(oe) (oe) oo n 12 
Ji roa a 
I= ee - = |} dy, dy,---dy 
hs, [2 d -) yj dy3 +++ dy, 

= Ne “exp (-35) dy, = (2n)"?(Atdy == an)?, (30.149) 
where we have used the standard integral pee exp(—ay?)dy = (a/a)'/? (see subsection 
6.4.2). From section 8.16, however, we note that the product of eigenvalues in (30.149) is 
equal to det V. Thus we finally obtain 


I = (2n)"?(det V)'/, 


and hence the normalisation in (30.148) ensures that f(x) integrates to unity. < 


The above example illustrates some importants points concerning the multi- 
variate Gaussian distribution. In particular, we note that the Y/ are independent 
Gaussian variables with mean zero and variance /;. Thus, given a general set of 
n Gaussian variables x with means w and covariance matrix V, one can always 
perform the above transformation to obtain a new set of variables y’, which are 
linear combinations of the old ones and are distributed as independent Gaussians 
with zero mean and variances 4;. 

This result is extremely useful in proving many of the properties of the mul- 
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tivariate Gaussian. For example, let us consider the quadratic form (multiplied 
by 2) appearing in the exponent of (30.148) and write it as 72, ie. 


hr = (= w)'V1(K = p). (30.150) 
From (30.149), we see that we may also write it as 
n y? 
2 pac 
Xn — hi > 


which is the sum of n independent Gaussian variables with mean zero and unit 
variance. Thus, as our notation implies, the quantity 72 is distributed as a chi- 
squared variable of order n. As illustrated in exercise 30.40, if the variables X; are 
required to satisfy m linear constraints of the form }~_, c:X; = 0 then 77 defined 
in (30.150) is distributed as a chi-squared variable of order n — m. 


30.16 Exercises 


30.1 By shading or numbering Venn diagrams, determine which of the following are 
valid relationships between events. For those that are, prove the relationship 
using de Morgan’s laws. 

(a) (XUY)=XNY. 
(b) XUY =(XUY). 
(c) (XUY)AZ =(XUZ)NY. 
(d) XUYAZ)=(XUY)NZ. 
(e) XUYAZ)=(XUY)UZ. 


30.2 Given that events X,Y and Z satisfy 


(XNY)JU(ZNX)U(XUY) =(ZUY)UL(ZUX)U(XNZ)NY}, 


prove that X > Y, and that either X NZ =@ or Y>Z. 

30.3 A and B each have two unbiased four-faced dice, the four faces being numbered 
1, 2, 3, 4. Without looking, B tries to guess the sum x of the numbers on the 
bottom faces of A’s two dice after they have been thrown onto a table. If the 
guess is correct B receives x? euros, but if not he loses x euros. 

Determine B’s expected gain per throw of A’s dice when he adopts each of the 
following strategies: 


(a) he selects x at random in the range 2< x < 8; 

(b) he throws his own two dice and guesses x to be whatever they indicate; 

(c) he takes your advice and always chooses the same value for x. Which number 
would you advise? 


30.4 Use the method of induction to prove equation (30.16), the probability addition 
law for the union of n general events. 

30.5 Two duellists, A and B, take alternate shots at each other, and the duel is over 
when a shot (fatal or otherwise!) hits its target. Each shot fired by A has a 
probability « of hitting B, and each shot fired by B has a probability 6 of hitting 
A. Calculate the probabilities P; and P>, defined as follows, that A will win such 
a duel: P;, A fires the first shot; P2, B fires the first shot. 

If they agree to fire simultaneously, rather than alternately, what is the proba- 
bility P; that A will win, ie. hit B without being hit himself? 
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30.6 


30.7 


30.8 


30.9 


30.10 


X,X,...,X, are independent, identically distributed, random variables drawn 
from a uniform distribution on [0, 1]. The random variables A and B are defined 
by 
A=min(Xj, Xo,..., Xn), B= max(X), X2,..., Xn). 
For any fixed k such thatO<k < } find the probability, p,, that both 
Ax<k and B>1-k. 


Check your general formula by considering directly the cases (a) k = 0, (b) k = 
(c)n =1 and (d) n=2. 

A tennis tournament is arranged on a straight knockout basis for 2” players, 
and for each round, except the final, opponents for those still in the competition 
are drawn at random. The quality of the field is so even that in any match it is 
equally likely that either player will win. Two of the players have surnames that 
begin with ‘Q’. Find the probabilities that they play each other 


(a) in the final, 
(b) at some stage in the tournament. 


i 
> 


This exercise shows that the odds are hardly ever ‘evens’ when it comes to dice 
rolling. 


(a) Gamblers A and B each roll a fair six-faced die, and B wins if his score is 
strictly greater than A’s. Show that the odds are 7 to 5 in A’s favour. 

(b) Calculate the probabilities of scoring a total T from two rolls of a fair die 
for T = 2,3,...,12. Gamblers C and D each roll a fair die twice and score 
respective totals Tc and Tp, D winning if Tp > Tc. Realising that the odds 
are not equal, D insists that C should increase her stake for each game. C 
agrees to stake £1.10 per game, as compared to D’s £1.00 stake. Who will 
show a profit? 


An electronics assembly firm buys its microchips from three different suppliers; 
half of them are bought from firm X, whilst firms Y and Z supply 30% and 
20%, respectively. The suppliers use different quality-control procedures and the 
percentages of defective chips are 2%, 4% and 4% for X, Y and Z, respectively. 
The probabilities that a defective chip will fail two or more assembly-line tests are 
40%, 60% and 80%, respectively, whilst all defective chips have a 10% chance 
of escaping detection. An assembler finds a chip that fails only one test. What is 
the probability that it came from supplier X? 

As every student of probability theory will know, Bayesylvania is awash with 
natives, not all of whom can be trusted to tell the truth, and lost, and apparently 
somewhat deaf, travellers who ask the same question several times in an attempt 
to get directions to the nearest village. 

One such traveller finds himself at a T-junction in an area populated by the 
Asciis and Bisciis in the ratio 11 to 5. As is well known, the Biscii always lie, but 
the Ascii tell the truth three quarters of the time, giving independent answers to 
all questions, even to immediately repeated ones. 


(a) The traveller asks one particular native twice whether he should go to the 
left or to the right to reach the local village. Each time he is told ‘left’. Should 
he take this advice, and, if he does, what are his chances of reaching the 
village? 

(b) The traveller then asks the same native the same question a third time, and 
for a third time receives the answer ‘left’. What should the traveller do now? 
Have his chances of finding the village been altered by asking the third 
question? 
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30.11 


30.12 


30.13 


30.14 


30.15 


30.16 


A boy is selected at random from amongst the children belonging to families with 
n children. It is known that he has at least two sisters. Show that the probability 
that he has k — 1 brothers is 


(n—1) 
(1 —m(k—Din—k)! 


for 1 < k <n—2 and zero for other values of k. Assume that boys and girls are 
equally likely. 

Villages A, B, C and D are connected by overhead telephone lines joining AB, 
AC, BC, BD and CD. As a result of severe gales, there is a probability p (the 
same for each link) that any particular link is broken. 


(a) Show that the probability that a call can be made from A to B is 
1— p*— 2p? +3p*—p’. 
(b) Show that the probability that a call can be made from D to A is 
1 — 2p? — 2p? + Sp* — 2p°. 


A set of 2N + 1 rods consists of one of each integer length 1,2,...,2N,2N +1. 
Three, of lengths a, b and c, are selected, of which a is the longest. By considering 
the possible values of b and c, determine the number of ways in which a non- 
degenerate triangle (ic. one of non-zero area) can be formed (i) if a is even, 
and (ii) if a is odd. Combine these results appropriately to determine the total 
number of non-degenerate triangles that can be formed with the 2N + 1 rods, 
and hence show that the probability that such a triangle can be formed from a 
random selection (without replacement) of three rods is 


(N —1)(4N +1) 
2(4N2 — 1) 


A certain marksman never misses his target, which consists of a disc of unit radius 
with centre O. The probability that any given shot will hit the target within a 
distance t of O is t?, for 0 <t < 1. The marksman fires n independendent shots 
at the target, and the random variable Y is the radius of the smallest circle with 
centre O that encloses all the shots. Determine the PDF for Y and hence find 
the expected area of the circle. 

The shot that is furthest from O is now rejected and the corresponding circle 
determined for the remaining n— 1 shots. Show that its expected area is 


n—1 
n+1 


The duration (in minutes) of a telephone call made from a public call-box is a 
random variable T. The probability density function of T is 


Tl. 


0 t<0, 
fo=44 0<rt<l1, 
ke t>1, 


where k is a constant. To pay for the call, 20 pence has to be inserted at the 
beginning, and a further 20 pence after each subsequent half-minute. Determine 
by how much the average cost of a call exceeds the cost of a call of average 
length charged at 40 pence per minute. 

Kittens from different litters do not get on with each other, and fighting breaks 
out whenever two kittens from different litters are present together. A cage 
initially contains x kittens from one litter and y from another. To quell the 
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30.17 


fighting, kittens are removed at random, one at a time, until peace is restored. 
Show, by induction, that the expected number of kittens finally remaining is 

x y 
eae ; 
y+1 x+1 


N(x, y) = 


If the scores in a cup football match are equal at the end of the normal period of 
play, a ‘penalty shoot-out’ is held in which each side takes up to five shots (from 
the penalty spot) alternately, the shoot-out being stopped if one side acquires an 
unassailable lead (i.e. has a lead greater than its opponents have shots remaining). 
If the scores are still level after the shoot-out a ‘sudden death’ competition takes 
place. 

In sudden death each side takes one shot and the competition is over if one 
side scores and the other does not; if both score, or both fail to score, a further 
shot is taken by each side, and so on. Team 1, which takes the first penalty, has 
a probability pi, which is independent of the player involved, of scoring and a 
probability gq; (= 1 — p:) of missing; pz and q are defined likewise. 

Define Pr(i : x,y) as the probability that team i has scored x goals after y 
attempts, and let f(M) be the probability that the shoot-out terminates after a 
total of M shots. 


(a) Prove that the probability that ‘sudden death’ will be needed is 
5 
F114) = S0OCP (rip2) (ain). 
r=0 


(b) Give reasoned arguments (preferably without first looking at the expressions 
involved) which show that 


2N-6 
od =: p2 Pr(1:r, N) Pr(2:5-— N +r, N— 1) 
fim = 2) = > { + qo Pr(1:6— N +r, N) Pr(2:r,N — 1) 


r= 


for N = 3,4,5 and 


2N-5 
a - pi Pr(1:5—N +r, N) Pr(2:r, N) 
fim =2N-+ 1) = of + aq Pr(l:r, N) Pr(2:5—N+r,N) 


r=0 


for N = 3,4. 
(c) Give an explicit expression for Pr(i:x,y) and hence show that if the teams 
are so well matched that p; = po = 1/2 then 


sa ame | N\(N — 1)!6 
FQN) =D) (==) TIN=HIG- NE HION 6ST 


r=0 


i a | (NIP 
fen+)= So (==) TIN=NIG-N+NIQN—S—n 


r=0 


(d) Evaluate these expressions to show that, expressing f(M) in units of 2-*, we 
have 


M 6 7 8 9 10) 11+ 
f(M) 8 24 42 56 63 63 


Give a simple explanation of why f(10) = f(11+). 
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30.18 


30.19 


30.20 


30.21 


30.22 


A particle is confined to the one-dimensional space 0 < x < a, and classically 
it can be in any small interval dx with equal probability. However, quantum 
mechanics gives the result that the probability distribution is proportional to 
sin’(ntx/a), where n is an integer. Find the variance in the particle’s position 
in both the classical and quantum-mechanical pictures, and show that, although 
they differ, the latter tends to the former in the limit of large n, in agreement 
with the correspondence principle of physics. 

A continuous random variable X has a probability density function f(x); the 
corresponding cumulative probability function is F(x). Show that the random 
variable Y = F(X) is uniformly distributed between 0 and 1. 

For a non-negative integer random variable X, in addition to the probability 
generating function ®y(t) defined in equation (30.71), it is possible to define the 
probability generating function 


WY x(t) = ys gat", 


n=0 


where g,, is the probability that X > n. 


(a) Prove that Dy and ‘Py are related by 
1— x(t) 
1-t 
(b) Show that E[X] is given by Wy(1) and that the variance of X can be 
expressed as 2'¥4,(1) + Px(1) — [Wx (1)P. 
(c) For a particular random variable X, the probability that X > n is equal to 
at! with 0 < « <1. Use the results in (b) to show that V[X] = «(1 — «)~?. 


Yx(t) = 


This exercise is about interrelated binomial trials. 


(a) In two sets of binomial trials T and t, the probabilities that a trial has a 
successful outcome are P and p, respectively, with corresponding probabilites 
of failure of Q = 1— P and q = 1—p. One ‘game’ consists of a trial T, 
followed, if T is successful, by a trial t and then a further trial T. The two 
trials continue to alternate until one of the T-trials fails, at which point the 
game ends. The score S for the game is the total number of successes in the 
t-trials. Find the PGF for S and use it to show that 


Pp Pp(1—Paq) 
—_ V[sS] = —————.. 
oe Me @ 


(b) Two normal unbiased six-faced dice A and B are rolled alternately starting 
with A; if A shows a 6 the experiment ends. If B shows an odd number no 
points are scored, if it shows a 2 or a 4 then one point is scored, whilst if 
it records a 6 then two points are awarded. Find the average and standard 
deviation of the score for the experiment and show that the latter is the 
greater. 


E[S] 


Use the formula obtained in subsection 30.8.2 for the moment generating function 
of the geometric distribution to determine the CGF, K,,(t), for the number of 
trials needed to record n successes. Evaluate the first four cumulants, and use 
them to confirm the stated results for the mean and variance, and to show that 
the distribution has skewness and kurtosis given, respectively, by 


ae 7, 2 
2—P sade ea OPS, 


ntl —p) n(1— p) 
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30.23 


30.24 


30.25 


30.26 


30.27 


30.28 


30.29 


30.30 


30.31 


A point P is chosen at random on the circle x? + y? = 1. The random variable 
X denotes the distance of P from (1,0). Find the mean and variance of X and 
the probability that X is greater than its mean. 

As assistant to a celebrated and imperious newspaper proprietor, you are given 
the job of running a lottery, in which each of his five million readers will have an 
equal independent chance, p, of winning a million pounds; you have the job of 
choosing p. However, if nobody wins it will be bad for publicity, whilst if more 
than two readers do so, the prize cost will more than offset the profit from extra 
circulation — in either case you will be sacked! Show that, however you choose 
p, there is more than a 40% chance you will soon be clearing your desk. 

The number of errors needing correction on each page of a set of proofs follows 
a Poisson distribution of mean yu. The cost of the first correction on any page is 
a and that of each subsequent correction on the same page is f. Prove that the 
average cost of correcting a page is 


a+ B(u—1)—(a— Bye™. 


In the game of Blackball, at each turn Muggins draws a ball at random from a 
bag containing five white balls, three red balls and two black balls; after being 
recorded, the ball is replaced in the bag. A white ball earns him $1, whilst a 
red ball gets him $2; in either case, he also has the option of leaving with his 
current winnings or of taking a further turn on the same basis. If he draws a 
black ball the game ends and he loses all he may have gained previously. Find 
an expression for Muggins’ expected return if he adopts the strategy of drawing 
up to n balls, provided he has not been eliminated by then. 

Show that, as the entry fee to play is $3, Muggins should be dissuaded from 
playing Blackball, but, if that cannot be done, what value of n would you advise 
him to adopt? 

Show that, for large r, the value at the maximum of the PDF for the gamma 
distribution of order r with parameter 4 is approximately 2/,/2x(r — 1). 

A husband and wife decide that their family will be complete when it includes 
two boys and two girls — but that this would then be enough! The probability 
that a new baby will be a girl is p. Ignoring the possibility of identical twins, 
show that the expected size of their family is 


1 
2(—-1- ; 
(55 ra) 
where q=1-—p. 


The probability distribution for the number of eggs in a clutch is Po(/), and the 
probability that each egg will hatch is p (independently of the size of the clutch). 
Show by direct calculation that the probability distribution for the number of 
chicks that hatch is Po(Ap) and so justify the assumptions made in the worked 
example at the end of subsection 30.7.1. 
A shopper buys 36 items at random in a supermarket, where, because of the 
sales tax imposed, the final digit (the number of pence) in the price is uniformly 
and randomly distributed from 0 to 9. Instead of adding up the bill exactly, 
she rounds each item to the nearest 10 pence, rounding up or down with equal 
probability if the price ends in a ‘5S’. Should she suspect a mistake if the cashier 
asks her for 23 pence more than she estimated? 
Under EU legislation on harmonisation, all kippers are to weigh 0.2000 kg, and 
vendors who sell underweight kippers must be fined by their government. The 
weight of a kipper is normally distributed, with a mean of 0.2000 kg and a 
standard deviation of 0.0100 kg. They are packed in cartons of 100 and large 
quantities of them are sold. 

Every day, a carton is to be selected at random from each vendor and tested 
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30.32 


30.33 


30.34 


according to one of the following schemes, which have been approved for the 
purpose. 


(a) The entire carton is weighed, and the vendor is fined 2500 euros if the 
average weight of a kipper is less than 0.1975 kg. 

(b) Twenty-five kippers are selected at random from the carton; the vendor is 
fined 100 euros if the average weight of a kipper is less than 0.1980 kg. 

(c) Kippers are removed one at a time, at random, until one has been found 
that weighs more than 0.2000 kg; the vendor is fined 4n(n — 1) euros, where 
n is the number of kippers removed. 


Which scheme should the Chancellor of the Exchequer be urging his government 
to adopt? 

In a certain parliament, the government consists of 75 New Socialites and 
the opposition consists of 25 Preservatives. Preservatives never change their 
mind, always voting against government policy without a second thought; New 
Socialites vote randomly, but with probability p that they will vote for their party 
leader’s policies. 

Following a decision by the New Socialites’ leader to drop certain manifesto 

commitments, N of his party decide to vote consistently with the opposition. The 
leader’s advisors reluctantly admit that an election must be called if N is such 
that, at any vote on government policy, the chance of a simple majority in favour 
would be less than 80%. Given that p = 0.8, estimate the lowest value of N that 
would precipitate an election. 
A practical-class demonstrator sends his twelve students to the storeroom to 
collect apparatus for an experiment, but forgets to tell each which type of 
component to bring. There are three types, A, B and C, held in the stores (in 
large numbers) in the proportions 20%, 30% and 50%, respectively, and each 
student picks a component at random. In order to set up one experiment, one 
unit each of A and B and two units of C are needed. Let Pr(N) be the probability 
that at least N experiments can be set up. 


(a) Evaluate Pr(3). 
(b) Find an expression for Pr(V) in terms of k, and kz, the numbers of compo- 


nents of types A and B respectively selected by the students. Show that Pr(2) 
can be written in the form 


6 8-i 
Pr(2) = (0.5) > 2c, (0.4)! PS 2IC; (0.6). 
i=2 j=2 


(c) By considering the conditions under which no experiments can be set up, 
show that Pr(1) = 0.9145. 


The random variables X and Y take integer values, x and y, both > 1, and such 
that 2x + y < 2a, where a is an integer greater than 1. The joint probability 
within this region is given by 
PriX =x, Y =y)=c(2x+y), 

where c is a constant, and it is zero elsewhere. 

Show that the marginal probability Pr(X = x) is 
6(a — x)(2x + 2a +4 1) 

a(a—1)(8a+5) ” 
and obtain expressions for Pr(Y = y), (a) when y is even and (b) when y is odd. 
Show further that 


Pr(X = x) 


6a°7+4a+1 


se a PTE 
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30.35 


30.36 


30.37 


30.38 


30.39 


30.40 


[You will need the results about series involving the natural numbers given in 
subsection 4.2.5. ] 

The continuous random variables X and Y have a joint PDF proportional to 
xy(x — y? with 0 < x < 1 and 0 < y < 1. Find the marginal distributions 
for X and Y and show that they are negatively correlated with correlation 
coefficient 2, 

A discrete random variable X takes integer values n = 0,1,...,N with probabil- 
ities p,. A second random variable Y is defined as Y = (X — y)*, where p is the 
expectation value of X. Prove that the covariance of X and Y is given by 


N N 
Cov[X, Y] = > Dn 3n>> py + 2p. 


n=0 n=0 


Now suppose that X takes all of its possible values with equal probability, and 
hence demonstrate that two random variables can be uncorrelated, even though 
one is defined in terms of the other. 

Two continuous random variables X and Y have a joint probability distribution 


f(x,y) = A(x? +’), 


where A is a constant and 0 < x < a,0 < y < a. Show that X and Y are negatively 
correlated with correlation coefficient —15/73. By sketching a rough contour 
map of f(x,y) and marking off the regions of positive and negative correlation, 
convince yourself that this (perhaps counter-intuitive) result is plausible. 
A continuous random variable X is uniformly distributed over the interval [—c, c]. 
A sample of 2n + 1 values of X is selected at random and the random variable 
Z is defined as the median of that sample. Show that Z is distributed over [—c, c] 
with probability density function 
(Q2n+1)! 5, 5, 
ful) = Gee (C2) 


Find the variance of Z. 
Show that, as the number of trials n becomes large but np; = /;, i = 1,2,...,k—1, 
remains finite, the multinomial probability distribution (30.146), 


n! : 

Xy x2 Xk 

M,(%1, X2,...5Xk) = - TPi Po Pies 
++ x4! 


- xX, !x5! ° 
can be approximated by a multiple Poisson distribution with k — 1 factors: 


k-1 eri Xi 

a i 

M)(X1,X25--+5Xk-1) = | | . 
ni x;! 
in 


(Write ee p; = 0 and express all terms involving subscript k in terms of n and 
6, either exactly or approximately. You will need to use n! = n‘[(n — e)!] and 
(1 —a/n)" = e~ for large n.) 


(a) Verify that the terms of M!, when summed over all values of x1, X2,...,X%—1 
add up to unity. 

(b) If k = 7 and A; = 9 for all i = 1,2,...,6, estimate, using the appropriate 
Gaussian approximation, the chance that at least three of x;, X2,...,X6 will 
be 15 or greater. 


The variables X;, i = 1,2,...,n, are distributed as a multivariate Gaussian, with 
means py; and a covariance matrix V. If the X; are required to satisfy the linear 
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30.1 
30.3 


30.5 
30.7 


30.9 
30.11 


30.13 


30.15 


30.17 


30.19 
30.21 


constraint pe c:X; = 0, where the c; are constants (and not all equal to zero), 
show that the variable 


tn = (K— WV T(x — p) 


follows a chi-squared distribution of order n — 1. 


30.17 Hints and answers 


(a) Yes, (b) no, (c) no, (d) no, (e) yes. 

Show that, if p,/16 is the probability that the total will be x, then the corrsponding 
gain is [p,(x* + x) — 16x]/16. (a) A loss of 0.36 euros; (b) a gain of 27/64 euros; 
(c) a gain of 2.5 euros, provided he takes your advice and guesses ‘5’ each time. 
Pi = a(x + B — of)- '. Py =a(1—f)(a+ Pf —aB)!; P3 = Pp. 

If p, is the probability that before the rth round both players are still in the 
tournament (and therefore have not met each other), show that 


1 gntl—r Ey) 1 rl gn+l—r —1 
Prot = qe” and hence that P= (3) et. 
(a) The probability that they meet in the final is p, = 2-"—)(2" —1)71. 
(b) The probability that they meet at some stage in the tournament is given by 
the sum S>"_, p,(2"F" = 1)-h = 2-0), 


The relative probabilities are X : Y :Z = 50 : 36:8 (in units of 107+); 25/47. 
Take A; as the event that a family consists of j boys and n — j girls, and B as 
the event that the boy has at least two sisters. Apply Bayes’ theorem. 

(i) For a even, the number of ways is 1+3+5+---+(a— 3), and (ii) for a odd 
itis2+4+6+---+(a—3). Combine the results for a = 2m and a = 2m+ 1, 
with m running from 2 to N, to show that the total number of non-degenerate 
triangles is given by N(4N + 1)(N — 1)/6. The number of possible selections of a 
set of three rods is (2N + 1)(2N)(2N — 1)/6. 

Show that k = e? and that the average duration of a call is 1 minute. Let p, 
be the probability that the call ends during the interval 0.5(n — 1) < t < 0.5n 


and c, = 20n be the corresponding cost. Prove that p;) = p. = 1 and that 


4 
Pn = 4e*(e — 1)e~", for n > 3. It follows that the average cost is 


30 —1) ‘. 
E[C] = 2 +2029 es ; 
The arithmetico-geometric series has sum (3e~! — 2e~?)/(e — 1)° and the total 


charge is 5(e + 1)/(e — 1) = 10.82 pence more than the 40 pence a uniform rate 

would cost. 

(a) The scores must be equal, at r each, after five attempts each. 

(b) M can only be even if team 2 gets too far ahead (or drops too far behind) to 

be caught (or catch up), with conditional probability p2 (or q2). Conversely, M 

can only be odd as a result of a final action by team 1. 

(c) Pr(i:x, y) ="Cxpiq; ~ 

(d) If the match is still alive at the tenth kick, team 2 is just as likely to lose it 

as to take it into sudden death. 

Show that dY /dX = f and use g(y) = f(x)|dx/dy|. 

(a) Use result (30.84) to show that the PGF for S is Q/(1— Pq — Ppt). Then use 
equations (30.74) and (30.76). 

(b) The PGF for the score is 6/(21 — 10¢ — St”) and the average score is 10/3. 
The variance is 145/9 and the standard deviation is 4.01. 
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30.25 
30.27 


30.29 
30.31 


30.33 
30.35 


30.37 


30.39 


Mean = 4/z. Variance = 2 — (16/z*). Probability that X exceeds its mean 
= 1—(2/n) sin '(2/n) = 0.561. 
Consider, separately, 0, 1 and > 2 errors on a page. 
Show that the maximum occurs at x = (r — 1)/A, and then use Stirling’s approx- 
imation to find the maximum value. 
Pr(k chicks hatching) = 7°, Po(n, 4) Bin(u, p). 
There is not much to choose between the schemes. In (a) the critical value of 
the standard variable is —2.5 and the average fine would be 15.5 euros. For 
(b) the corresponding figures are —1.0 and 15.9 euros. Scheme (c) is governed 
by a geometric distribution with p = q = ; and leads to an expected fine 
of oe 1 An(n — 1)(5)". The sum can be evaluated by differentiating the result 
e p/(1 — p) with respect to p, and gives the expected fine as 16 euros. 
(a) [ Tonos (0.2)7]/(6! 3!3!) = 0.0624. 
You will need to establish the normalisation constant for the distribution (36), 
the common mean value (3/5) and the common standard deviation (3/10). The 
marginal distributions are f(x) = 3x(6x? — 8x + 3), and the same function of y. 
The covariance has the value —3/50, yielding a correlation of —2/3. 
= 3/(24a*); wy = wy = 5a/8; o% = of = 73a’/960; E[XY] = 3a’/8; 
Cov[X, Y] = —a’/64. 
(b) With the continuity correction Pr(x; > 15) = 0.0334. The probability that at 
least three are 15 or greater is 7.5 x 107+. 
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In this chapter, we turn to the study of statistics, which is concerned with 
the analysis of experimental data. In a book of this nature we cannot hope 
to do justice to such a large subject; indeed, many would argue that statistics 
belongs to the realm of experimental science rather than in a mathematics 
textbook. Nevertheless, physical scientists and engineers are regularly called upon 
to perform a statistical analysis of their data and to present their results in a 
statistical context. Therefore, we will concentrate on this aspect of a much more 
extensive subject 


31.1 Experiments, samples and populations 


We may regard the product of any experiment as a set of N measurements of some 
quantity x or set of quantities x, y,...,z. This set of measurements constitutes the 
data. Each measurement (or data item) consists accordingly of a single number x; 
or a set of numbers (Xj, yi,...,,2:), Where i = 1,...,,N. For the moment, we will 
assume that each data item is a single number, although our discussion can be 
extended to the more general case. 

As a result of inaccuracies in the measurement process, or because of intrinsic 
variability in the quantity x being measured, one would expect the N measured 
values x1, X2,...,Xy to be different each time the experiment is performed. We may 


S There are, in fact, two separate schools of thought concerning statistics: the frequentist approach 
and the Bayesian approach. Indeed, which of these approaches is the more fundamental is still a 
matter of heated debate. Here we shall concentrate primarily on the more traditional frequentist 
approach (despite the preference of some of the authors for the Bayesian viewpoint!). For a fuller 
discussion of the frequentist approach one could refer to, for example, A. Stuart and K. Ord, 
Kendall’s Advanced Theory of Statistics, vol. 1 (London: Edward Arnold, 1994) or J. F. Kenney 
and E. S. Keeping, Mathematics of Statistics (New York: Van Nostrand, 1954). For a discussion 
of the Bayesian approach one might consult, for example, D. S. Sivia, Data Analysis: A Bayesian 
Tutorial (Oxford: Oxford University Press, 1996). 
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therefore consider the x; as a set of N random variables. In the most general case, 
these random variables will be described by some N-dimensional joint probability 
density function P(x, x2,... ,xn).§ In other words, an experiment consisting of N 
measurements is considered as a single random sample from the joint distribution 
(or population) P(x), where x denotes a point in the N-dimensional data space 
having coordinates (x1, X2,...,XN)- 

The situation is simplified considerably if the sample values x; are independent. 
In this case, the N-dimensional joint distribution P(x) factorises into the product 
of N one-dimensional distributions, 


P(x) = P(x1)P(x2)--- P(xy). (31.1) 


In the general case, each of the one-dimensional distributions P(x;) may be 
different. A typical example of this occurs when N independent measurements 
are made of some quantity x but the accuracy of the measuring procedure varies 
between measurements. 

It is often the case, however, that each sample value x; is drawn independently 
from the same population. In this case, P(x) is of the form (31.1), but, in addition, 
P(x;) has the same form for each value of i. The measurements x1, X2,...,xN 
are then said to form a random sample of size N from the one-dimensional 
population P(x). This is the most common situation met in practice and, unless 
stated otherwise, we will assume from now on that this is the case. 


31.2 Sample statistics 


Suppose we have a set of N measurements x1, X2,...,xy. Any function of these 
measurements (that contains no unknown parameters) is called a sample statistic, 
or often simply a statistic. Sample statistics provide a means of characterising the 
data. Although the resulting characterisation is inevitably incomplete, it is useful 
to be able to describe a set of data in terms of a few pertinent numbers. We now 
discuss the most commonly used sample statistics. 


S In this chapter, we will adopt the common convention that P(x) denotes the particular probability 
density function that applies to its argument, x. This obviates the need to use a different letter 
for the PDF of each new variable. For example, if X¥ and Y are random variables with different 
PDFs, then properly one should denote these distributions by f(x) and g(y), say. In our shorthand 
notation, these PDFs are denoted by P(x) and P(y), where it is understood that the functional 
form of the PDF may be different in each case. 
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188.7 204.7 193.2 169.0 
168.1 189.8 166.3 200.0 


Table 31.1 Experimental data giving eight measurements of the round trip 


time in milliseconds for a computer ‘packet’ to travel from Cambridge UK to 
Cambridge MA. 


31.2.1 Averages 


The simplest number used to characterise a sample is the mean, which for N 
values x;, i = 1,2,...,N, is defined by 


1 N 
X= NL (31.2) 
= 


In words, the sample mean is the sum of the sample values divided by the number 
of values in the sample. 


Using (31.2) the sample mean in milliseconds is given by 


X = $(188.7 + 204.7 + 193.2 + 169.0 + 168.1 + 189.8 + 166.3 + 200.0) 


_ 14798 _ se4ors, 


Since the sample values in table 31.1 are quoted to an accuracy of one decimal place, it is 
usual to quote the mean to the same accuracy, i.e. as X = 185.0. < 


Strictly speaking the mean given by (31.2) is the arithmetic mean and this is by 
far the most common definition used for a mean. Other definitions of the mean 
are possible, though less common, and include 


N 1/N 
Xp = (11 “) ; (31.3) 


(i) the geometric mean, 


(11) the harmonic mean, 


N 
y= (31.4) 
Viet L/xi 
(ili) the root mean square, 
1/2 
NY 5) 
Xims = (=) (31.5) 


STATISTICS 


It should be noted that, xX, X, and X;ms would remain well defined even if some 
sample values were negative, but the value of xX, could then become complex. 
The geometric mean should not be used in such cases. 


> Calculate X,, X, and Xrms for the sample given in table 31.1. 


The geometric mean is given by (31.3) to be 


X_ = (188.7 x 204.7 x --- x 200.0)!/8 = 184.4. 


The harmonic mean is given by (31.4) to be 
x g 183.9 
“bh (1/188.7) + (1/204.7) +--+ + (1/200.0) a 


Finally, the root mean square is given by (31.5) to be 


Xams = [4(188.7? + 204.7? +--+ + 200.07)] '/? = 185.5. < 


Two other measures of the ‘average’ of a sample are its mode and median. The 
mode is simply the most commonly occurring value in the sample. A sample may 
possess several modes, however, and thus it can be misleading in such cases to 
use the mode as a measure of the average of the sample. The median of a sample 
is the halfway point when the sample values x; (i = 1,2,...,N) are arranged in 
ascending (or descending) order. Clearly, this depends on whether the size of 
the sample, N, is odd or even. If N is odd then the median is simply equal to 
X(N+1)/2> Whereas if N is even the median of the sample is usually taken to be 


5(xn/2 + Xw/2)41): 


> Find the mode and median of the sample given in table 31.1. 


From the table we see that each sample value occurs exactly once, and so any value may 
be called the mode of the sample. 

To find the sample median, we first arrange the sample values in ascending order and 
obtain 


166.3, 168.1, 169.0, 188.7, 189.8, 193.2, 200.0, 204.7. 
Since the number of sample values N = 8, which is even, the median of the sample is 


5 (4 + Xs) = 5(188.7 + 189.8) = 189.25. < 


31.2.2 Variance and standard deviation 


The variance and standard deviation both give a measure of the spread of values 
in a sample about the sample mean x. The sample variance is defined by 


1a 
2_ ey? 
s= a 2 (x; — Xx), (31.6) 
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and the sample standard deviation is the positive square root of the sample 
variance, Le. 


(31.7) 


> Find the sample variance and sample standard deviation of the data given in table 31.1. 


We have already found that the sample mean is 185.0 to one decimal place. However, 
when the mean is to be used in the subsequent calculation of the sample variance it is 
better to use the most accurate value available. In this case the exact value is 184.975, and 
so using (31.6), 


v= : [(188.7 — 184.975)” +++. + (200.0 — 184.975)’] 
_ 1608.36 


= 201.0, 


where once again we have quoted the result to one decimal place. The sample standard 
deviation is then given by s = \/201.0 = 14.2. As it happens, in this case the difference 
between the true mean and the rounded value is very small compared with the variation 
of the individual readings about the mean and using the rounded value has a negligible 
effect; however, this would not be so if the difference were comparable to the sample 
standard deviation. < 


Using the definition (31.7), it is clear that in order to calculate the standard 
deviation of a sample we must first calculate the sample mean. This requirement 
can be avoided, however, by using an alternative form for s*. From (31.6), we see 
that 


1 N 
ee ys 2 
s=—Y) (x;—X) 
No 
ee 1 ie 
= Dt ye ete ee 
i=1 i=1 i=1 
= x2 — 25745? = 2? —¥? 


We may therefore write the sample variance s as 


N N 2 
2_W_ 22 Sox (z2*] (31.8) 
S=x-xX= X; xi}, : 
Nes Nee 


from which the sample standard deviation is found by taking the positive square 
root. Thus, by evaluating the quantities pa x; and baa x? for our sample, we 
can calculate the sample mean and sample standard deviation at the same time. 
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> Calculate \~”, x; and )~™, x? for the data given in table 31.1 and hence find the mean 
and standard deviation of the sample. 


From table 31.1, we obtain 


N 
a x; = 188.7 + 204.7 + --- + 200.0 = 1479.8, 


N 
> x7 = (188.7)° + (204.7)? +--+ + (200.0)? = 275 334.36. 


Since N = 8, we find as before (quoting the final results to one decimal place) 


2 
oa “a = 185.0, oes 2 ass _ (*) dee 


31.2.3: Moments and central moments 


By analogy with our discussion of probability distributions in section 30.5, the 
sample mean and variance may also be described respectively as the first moment 
and second central moment of the sample. In general, for a sample x;, i = 


1,2,...,N, we define the rth moment m, and rth central moment n, as 
ix 
Mr = ye (31.9) 
i=l 
iw 
hay Soi — mi)". (31.10) 


Thus the sample mean xX and variance s? 


may also be written as m, and no 
respectively. As is common practice, we have introduced a notation in which 
a sample statistic is denoted by the Roman letter corresponding to whichever 
Greek letter is used to describe the corresponding population statistic. Thus, we 
use m, and n, to denote the rth moment and central moment of a sample, since 
in section 30.5 we denoted the rth moment and central moment of a population 
by yu, and v, respectively. 

This notation is particularly useful, since the rth central moment of a sample, 
m,, may be expressed in terms of the rth- and lower-order sample moments n, in a 
way exactly analogous to that derived in subsection 30.5.5 for the corresponding 
population statistics. As discussed in the previous section, the sample variance is 
given by s? = x2 — X? but this may also be written as ny = m)— m;, which is to be 
compared with the corresponding relation v. = 2—p{ derived in subsection 30.5.3 
for population statistics. This correspondence also holds for higher-order central 
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moments of the sample. For example, 


1 N 
m= 2 —m) 
= 


1 

=H y (x? — 3m,x? + 3m7x; — m3 
i=1 

= 2 3 

= m3 — 3mm + 3mm, — mj; 


= m3 — 3mm + 2m?, (31.11) 


which may be compared with equation (30.53) in the previous chapter. 
Mirroring our discussion of the normalised central moments », of a population 
in subsection 30.5.5, we can also describe a sample in terms of the dimensionless 
quantities 
Nk nk 
8k = ie = e 


g3 and gy, are called the sample skewness and kurtosis. Likewise, it is common to 
define the excess kurtosis of a sample by g4 — 3. 


31.2.4 Covariance and correlation 


So far we have assumed that each data item of the sample consists of a single 
number. Now let us suppose that each item of data consists of a pair of numbers, 
so that the sample is given by (x;, y;), i= 1,2,...,N. 

We may calculate the sample means, x and j, and sample variances, s2 and 
s*, of the x; and y; values individually but these statistics do not provide any 
measure of the relationship between the x; and y;. By analogy with our discussion 
in subsection 30.12.3 we measure any interdependence between the x; and y; in 
terms of the sample covariance, which is given by 


1 N 
Vy =H di — x\(yi — J) 


=(x—x)(y— J) 
= xy — Xp. (31.12) 


Writing out the last expression in full, we obtain the form most useful for 
calculations, which reads 
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Nyy = 0.0 Nyy = 0.1 Nyy = 0.5 


Figure 31.1 Scatter plots for two-dimensional data samples of size N = 1000, 
with various values of the correlation r. No scales are plotted, since the value 
of r is unaffected by shifts of origin or changes of scale in x and y. 


We may also define the closely related sample correlation by 


Vey 
, 
Sy Sy 


which can take values between —1 and +1. If the x; and y; are independent then 
Vyy = 0 = ry, and from (31.12) we see that xy = xy. It should also be noted 
that the value of ry, is not altered by shifts in the origin or by changes in the 
scale of the x; or y;. In other words, if x’ = ax +b and y’ = cy +d, where a, 
b, c, d are constants, then ryy = rxy. Figure 31.1 shows scatter plots for several 
two-dimensional random samples x;, y; of size N = 1000, each with a different 
value of ry. 


>» Ten UK citizens are selected at random and their heights and weights are found to be as 
follows (to the nearest cm or kg respectively): 


Person ADP eiBY RICO 8 AD Rein MOC Rot EG ae stig Bus ey 
Height (cm) 194 168 177 180 171 190 151 169 175 182 
Weight (kg) 75 53 72 80 75 75 57 67 46 68 


Calculate the sample correlation between the heights and weights. 


In order to find the sample correlation, we begin by calculating the following sums (where 
x; are the heights and y; are the weights) 


$7 x = 1757, > y¥; = 668, 
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So x7 = 310041, So y7 = 45746, SY” xiyi = 118.029. 
The sample consists of N = 10 pairs of numbers, so the means of the x; and of the y; are 


given by X = 175.7 and j = 66.8. Also, Xy = 11802.9. Similarly, the standard deviations 
of the x; and y; are calculated, using (31.8), as 


310041 1757? 
ay Se EY tie 
45 746 668\7 


Thus the sample correlation is given by 
: xy—xy — 11802.9 — (175.7)(66.8) 
a SxSy (11.6)(10.6) 


Thus there is a moderate positive correlation between the heights and weights of the 
people measured. < 


0.54. 


It is straightforward to generalise the above discussion to data samples of 
arbitrary dimension, the only complication being one of notation. We choose 
to denote the ith data item from an n-dimensional sample as ee es), 
where the bracketted superscript runs from 1 to n and labels the elements within 
a given data item whereas the subscript i runs from 1 to N and labels the data 
items within the sample. In this n-dimensional case, we can define the sample 


covariance matrix whose elements are 


Vg = xOxO — x6 50 


and the sample correlation matrix with elements 
Ver 


r,y = — 


SKS , 


Both these matrices are clearly symmetric but are not necessarily positive definite. 


31.3. Estimators and sampling distributions 


In general, the population P(x) from which a sample x1,x2,...,xv 1s drawn 
is unknown. The central aim of statistics is to use the sample values x; to infer 
certain properties of the unknown population P(x), such as its mean, variance and 
higher moments. To keep our discussion in general terms, let us denote the various 
parameters of the population by aj, a,..., or collectively by a. Moreover, we make 
the dependence of the population on the values of these quantities explicit by 
writing the population as P(x|a). For the moment, we are assuming that the 
sample values x; are independent and drawn from the same (one-dimensional) 
population P(x|a), in which case 


P(x\a) = P(x;|a)P(xa|a)--- P(xyla). 
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Suppose, we wish to estimate the value of one of the quantities a1, a2,..., which 
we will denote simply by a. Since the sample values x; provide our only source of 
information, any estimate of a must be some function of the x;, i.e. some sample 
statistic. Such a statistic is called an estimator of a and is usually denoted by a(x), 
where x denotes the sample elements x1, X2,...,Xy- 

Since an estimator @ is a function of the sample values of the random variables 
X1,X2,...,Xy, it too must be a random variable. In other words, if a number of 
random samples, each of the same size N, are taken from the (one-dimensional) 
population P(x|a) then the value of the estimator a will vary from one sample to 
the next and in general will not be equal to the true value a. This variation in the 
estimator is described by its sampling distribution P(@|a). From section 30.14, this 
is given by 


P(dja) da = P(x|a) dx, 


where dx is the infinitesimal ‘volume’ in x-space lying between the ‘surfaces’ 
a(x) = @ and a(x) = 4+ da. The form of the sampling distribution generally 
depends upon the estimator under consideration and upon the form of the 
population from which the sample was drawn, including, as indicated, the true 
values of the quantities a. It is also usually dependent on the sample size N. 


>The sample values x\,X2,...,xn are drawn independently from a Gaussian distribution 


with mean « and variance o. Suppose that we choose the sample mean X as our estimator 
ft of the population mean. Find the sampling distributions of this estimator. 


The sample mean x is given by 
hat Dy Peta 
X= (x1 +X). +--+ +x 
we 2 N)s 
where the x; are independent random variables distributed as x; ~ N(,07). From our 
discussion of multiple Gaussian distributions on page 1189, we see immediately that x will 
also be Gaussian distributed as N(,07/N). In other words, the sampling distribution of 


X is given by 


P(x|,0) = 


=a (31.13) 


1 
SS CX i) 
\/2n0?/N . 207/N 


Note that the variance of this distribution is o?/N. < 


31.3.1 Consistency, bias and efficiency of estimators 


For any particular quantity a, we may in fact define any number of different 
estimators, each of which will have its own sampling distribution. The quality 
of a given estimator @ may be assessed by investigating certain properties of its 
sampling distribution P(G|a). In particular, an estimator @ is usually judged on 
the three criteria of consistency, bias and efficiency, each of which we now discuss. 
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Consistency 
An estimator @ is consistent if its value tends to the true value a in the large-sample 
limit, ie. 

lim @=a. 

Noo 
Consistency is usually a minimum requirement for a useful estimator. An equiv- 
alent statement of consistency is that in the limit of large N the sampling 
distribution P(d|a) of the estimator must satisfy 


Jim Pala) — 6(a—a). 


Bias 


The expectation value of an estimator @ is given by 
E[a] = [ ertaiayaa = J ao Ptsia) ds, (31.14) 


where the second integral extends over all possible values that can be taken by 
the sample elements x1, x2,...,x. This expression gives the expected mean value 
of a from an infinite number of samples, each of size N. The bias of an estimator 
@ is then defined as 


b(a) = E[4] — a. (31.15) 


We note that the bias b does not depend on the measured sample values 
X1,X2,...,Xy. In general, though, it will depend on the sample size N, the func- 
tional form of the estimator @ and, as indicated, on the true properties a of 
the population, including the true value of a itself. If b = 0 then @ is called an 
unbiased estimator of a. 


> An estimator @ is biased in such a way that E[@] = a-+b(a), where the bias b(a) is given 


by (bj —1)a+ bz and b, and bp are known constants. Construct an unbiased estimator of a. 


Let us first write E[G] is the clearer form 


E[a] at (by 1)a+ by bia | bo. 


The task of constructing an unbiased estimator is now trivial, and an appropriate choice 
is @ = (a4 —by)/b,, which (as required) has the expectation value 


Efa] = fi =a 
1 
Efficiency 


The variance of an estimator is given by 


V [a] = fo-za [@])?P (ala) da = [ioe [@])P(x|a) dx 
(31.16) 
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and describes the spread of values @ about E[@] that would result from a large 
number of samples, each of size N. An estimator with a smaller variance is said 
to be more efficient than one with a larger variance. As we show in the next 
section, for any given quantity a of the population there exists a theoretical lower 
limit on the variance of any estimator @. This result is known as Fisher’s inequality 
(or the Cramér—Rao inequality) and reads 


2 a2 
vine (ae 22) fe [Ee | (31.17) 
da 6a? 


where P stands for the population P(x|a) and b is the bias of the estimator. 
Denoting the quantity on the RHS of (31.17) by Vimnin, the efficiency e of an 
estimator is defined as 


oS Vinin/V [a]. 


An estimator for which e = 1 is called a minimum-variance or efficient estimator. 
Otherwise, if e < 1, @ is called an inefficient estimator. 


It should be noted that, in general, there is no unique ‘optimal’ estimator @ for 
a particular property a. To some extent, there is always a trade-off between bias 
and efficiency. One must often weigh the relative merits of an unbiased, inefficient 
estimator against another that is more efficient but slightly biased. Nevertheless, a 
common choice is the best unbiased estimator (BUE), which is simply the unbiased 
estimator @ having the smallest variance V [4]. 

Finally, we note that some qualities of estimators are related. For example, 
suppose that @ is an unbiased estimator, so that E[@] = a and V[a@] — 0 as 
N > o. Using the Bienaymé-Chebyshev inequality discussed in subsection 30.5.3, 
it follows immediately that @ is also a consistent estimator. Nevertheless, it does 
not follow that a consistent estimator is unbiased. 


>The sample values X1,X2,...,Xy are drawn independently from a Gaussian distribution 
with mean a and variance o. Show that the sample mean xX is a consistent, unbiased, 
minimum-variance estimator of |. 


We found earlier that the sampling distribution of x is given by 


1 (x — py)? 
Jinez[N 26?/N | 


from which we see immediately that E[x] = and V[X] = o?/N. Thus x is an unbiased 
estimator of 4. Moreover, since it is also true that V[x] > 0 as N — o, x is a consistent 
estimator of w. 

In order to determine whether X is a minimum-variance estimator of 4, we must use 
Fisher’s inequality (31.17). Since the sample values x; are independent and drawn from a 
Gaussian of mean p and standard deviation o, we have 


P(x|1,0) 


N 


In P(x|p,0) = as Se [inane zs (xi a 
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and, on differentiating twice with respect to wu, we find 
inp  N 
je 


This is independent of the x; and so its expectation value is also equal to —N/o?. With b 
set equal to zero in (31.17), Fisher’s inequality thus states that, for any unbiased estimator 
fi of the population mean, 
- 
Vif] > —. 
[2] = N 


Since V[X] = 0?/N, the sample mean x is a minimum-variance estimator of pu. < 


31.3.2 Fisher’s inequality 


As mentioned above, Fisher’s inequality provides a lower limit on the variance of 
any estimator a of the quantity a; it reads 


ab \? 2 
vies (ees) fel |. (31.18) 
oa oa 


where P stands for the population P(x|a) and b is the bias of the estimator. 
We now present a proof of this inequality. Since the derivation is somewhat 
complicated, and many of the details are unimportant, this section can be omitted 
on a first reading. Nevertheless, some aspects of the proof will be useful when 
the efficiency of maximum-likelihood estimators is discussed in section 31.5. 


> Prove Fisher’s inequality (31.18). 


The normalisation of P(x|a) is given by 


[ Posiaya’s =1, (31.19) 


where dNx = dx,dx,---dxy and the integral extends over all the allowed values of the 
sample items x;. Differentiating (31.19) with respect to the parameter a, we obtain 


i ax [rex 0. (31.20) 
oa oa 


We note that the second integral is simply the expectation value of 0 In P/éa, where the 
average is taken over all possible samples x;, i = 1,2,...,N. Further, by equating the two 
expressions for 0E[@]/da obtained by differentiating (31.15) and (31.14) with respect to a 
we obtain, dropping the functional dependencies, a second relationship, 


4 4 
t+ a fs aN x [a peeax (31.21) 


Oa 


Now, multiplying (31.20) by «(a), where «(a) is any function of a, and subtracting the 
result from (31.21), we obtain 


n OlInP ow ob 
[ee a(a)] aa Pd*x l+7. 


At this point we must invoke the Schwarz inequality proved in subsection 8.1.3. The proof 
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is trivially extended to multiple integrals and shows that for two real functions, g(x) and 


A(x), 
2 
(/ s(x) dx) ( [ 0a") > (/ atx)hts) ‘ (31.22) 


If we now let g = [a— a(a)]./P and h= (dln P/6a),/P, we find 


a 2 2 
(Je—snrres f(t) ve] = (048)! 


On the LHS, the factor in braces represents the expected spread of a-values around the 
point «(a). The minimum value that this integral may take occurs when «(a) = E[@]. 
Making this substitution, we recognise the integral as the variance V [4], and so obtain the 


result 
: ab\?[ f (amnP\?. y | 
V(@qjJ>(it+ 3a Aa Pd*‘x : (31.23) 


We note that the factor in brackets is the expectation value of (0 In P/éa)’. 
Fisher’s inequality is, in fact, often quoted in the form (31.23). We may recover the form 
(31.18) by noting that on differentiating (31.20) with respect to a we obtain 


@ InP OInPé@P\ 
ra 6a? an da me) as . 
Writing 0P/da as (6 In P/éa)P and rearranging we find that 
éinP\*>. @inP 
I( a) Pa’x=— aa Pd'x. 
Substituting this result in (31.23) gives 
apy 2 2 -1 
via>—(14+ 2 [eres 
da da? 


Since the factor in brackets is the expectation value of 671InP/da*, we have recovered 
result (31.18). < 


31.3.3 Standard errors on estimators 


For a given sample x1, x2,...,xy, we may calculate the value of an estimator a(x) 
for the quantity a. It is also necessary, however, to give some measure of the 
statistical uncertainty in this estimate. One way of characterising this uncertainty 
is with the standard deviation of the sampling distribution P(@|a), which is given 
simply by 


oq = (V[a])'?. (31.24) 


If the estimator a(x) were calculated for a large number of samples, each of size 
N, then the standard deviation of the resulting @ values would be given by (31.24). 
Consequently, oq is called the standard error on our estimate. 

In general, however, the standard error og depends on the true values of some 
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or all of the quantities a and they may be unknown. When this occurs, one must 
substitute estimated values of any unknown quantities into the expression for og 
in order to obtain an estimated standard error Gq. One then quotes the result as 


a=axGyq. 


> Ten independent sample values x;, i= 1,2,...,10, are drawn at random from a Gaussian 
distribution with standard deviation o = 1. The sample values are as follows (to two decimal 
places): 


2.22 2.56 1.07 0.24 018 0.95 073 —0.79 2.09 1.81 


Estimate the population mean 1, quoting the standard error on your result. 


We have shown in the final worked example of subsection 31.3.1 that, in this case, x is 
a consistent, unbiased, minimum-variance estimator of and has variance V[X] = o7/N. 
Thus, our estimate of the population mean with its associated standard error is 


Paes 914-059, 
JN 
If the true value of o had not been known, we would have needed to use an estimated 
value G in the expression for the standard error. Useful basic estimators of o are discussed 
in subsection 31.4.2. < 


It should be noted that the above approach is most meaningful for unbiased 
estimators. In this case, E[@] = a and so og describes the spread of G-values about 
the true value a. For a biased estimator, however, the spread about the true value 
a is given by the root mean square error €4, which is defined by 


ej = El(a—ay’] 
= E((a— E[a])"] + (E[a] — a) 
= V[a] + b(a)’. 


We see that €% is the sum of the variance of @ and the square of the bias and so 
can be interpreted as the sum of squares of statistical and systematic errors. For 
a biased estimator, it is often more appropriate to quote the result as 


a=ate,. 


As above, it may be necessary to use estimated values 4 in the expression for the 
root mean square error and thus to quote only an estimate éq of the error. 


31.3.4 Confidence limits on estimators 


An alternative (and often equivalent) way of quoting a statistical error is with a 
confidence interval. Let us assume that, other than the quantity of interest a, the 
quantities a have known fixed values. Thus we denote the sampling distribution 
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P (aja) 


a,(a) g(a) 


Figure 31.2 The sampling distribution P(a|a) of some estimator @ for a given 
value of a. The shaded regions indicate the two probabilities Pr(@ < G,(a)) = « 
and Pr(@ > ap(a)) = B. 


of @ by P(a|a). For any particular value of a, one can determine the two values 
a,(a) and ag(a) such that 
a, (a) 
Pr(a < a,(a)) = / P(aja) da = a, (31.25) 


—oo 
aa 


Pr(@ > Gp(a)) = | P (ala) da = B. (31.26) 


g(a) 


This is illustrated in figure 31.2. Thus, for any particular value of a, the probability 
that the estimator @ lies within the limits @,(a) and ag(a) is given by 


p(a) 
Pr(a,(a) < @ < Gig(a)) = / P(a@ja)da=1-—a-—f. 
a,(a) 
Now, let us suppose that from our sample x1, X2,...,xy, we actually obtain the 


value Gps for our estimator. If @ is a good estimator of a then we would expect 
@,(a) and Gg(a) to be monotonically increasing functions of a (ie. 4, and Gg both 
change in the same sense as a when the latter is varied). Assuming this to be the 
case, we can uniquely define the two numbers a_ and a, by the relationships 


Gy(4.) = obs and = ig(a_) = ops. 
From (31.25) and (31.26) it follows that 
Pr(a, <a)=«a and Pr(a_ > a) =f, 
which when taken together imply 
Pr(a_<a<a,)=1—a-f. (31.27) 


Thus, from our estimate ops, we have determined two values a_ and a, such that 
this interval contains the true value of a with probability 1 — « — f. It should be 
emphasised that a_ and a, are random variables. If a large number of samples, 
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P(aja_) P(aja,) 


A 
obs 


Figure 31.3. An illustration of how the observed value of the estimator, Gobs, 
and the given values « and f determine the two confidence limits a_ and a4, 
which are such that @,(a,) = Gps = 4g(a_). 


each of size N, were analysed then the interval [a_,a;] would contain the true 
value a on a fraction 1 — «— f of the occasions. 

The interval [a_,a,] is called a confidence interval on a at the confidence 
level 1—«—. The values a_ and a, themselves are called respectively the 
lower confidence limit and the upper confidence limit at this confidence level. In 
practice, the confidence level is often quoted as a percentage. A convenient way 
of presenting our results is 


Gobs 
‘i P(dla,) da =a, (31.28) 
/ " P(ala_)da = B. (31.29) 

obs 


The confidence limits may then be found by solving these equations for a_ and 
a4 either analytically or numerically. The situation is illustrated graphically in 
figure 31.3. 

Occasionally one might not combine the results (31.28) and (31.29) but use 
either one or the other to provide a one-sided confidence interval on a. Whenever 
the results are combined to provide a two-sided confidence interval, however, the 
interval is not specified uniquely by the confidence level 1 — «— . In other words, 
there are generally an infinite number of intervals [a_, a+] for which (31.27) holds. 
To specify a unique interval, one often chooses « = f, resulting in the central 
confidence interval on a. All cases can be covered by calculating the quantities 
c=a—a_ and d=a,—4 and quoting the result of an estimate as 


a=ate. 


So far we have assumed that the quantities a other than the quantity of interest 
a are known in advance. If this is not the case then the construction of confidence 
limits is considerably more complicated. This is discussed in subsection 31.3.6. 
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31.3.5 Confidence limits for a Gaussian sampling distribution 


An important special case occurs when the sampling distribution is Gaussian; if 
the mean is a and the standard deviation is o, then 
, 1 a—ay 
P(aja, oq) = exp | P 
2no; oF 


(31.30) 


For almost any (consistent) estimator a, the sampling distribution will tend to 
this form in the large-sample limit N — 00, as a consequence of the central limit 
theorem. For a sampling distribution of the form (31.30), the above procedure 
for determining confidence intervals becomes straightforward. Suppose, from our 
sample, we obtain the value Gp; for our estimator. In this case, equations (31.28) 
and (31.29) become 


where ®(z) is the cumulative probability function for the standard Gaussian distri- 
bution, discussed in subsection 30.9.1. Solving these equations for a_ and ax gives 


a_ = Apps — og® 1 (1 — B), (31.31) 
4, = Apps + og® (1 — «); (31.32) 
we have used the fact that @~!(«) = —®~!(1—«) to make the equations symmetric. 


The value of the inverse function ®—'(z) can be read off directly from table 30.3, 
given in subsection 30.9.1. For the normally used central confidence interval one 
has a = f. In this case, we see that quoting a result using the standard error, as 


a=aton, (31.33) 


is equivalent to taking ®—'(1—«) = 1. From table 30.3, we find « = 1— 0.8413 = 
0.1587, and so this corresponds to a confidence level of 1 — 2(0.1587) = 0.683. 
Thus, the standard error limits give the 68.3% central confidence interval. 


> Ten independent sample values x;, i= 1,2,...,10, are drawn at random from a Gaussian 
distribution with standard deviation ¢ = 1. The sample values are as follows (to two decimal 
places ): 


2.22 2.56 107 0.24 018 0.95 0.73 —0.79 2.09 1.81 


Find the 90% central confidence interval on the population mean ,L. 


Our estimator fi is the sample mean xX. As shown towards the end of section 31.3, the 
sampling distribution of x is Gaussian with mean E[X] and variance V[X] = o7/N. Since 
o = 1 in this case, the standard error is given by oy = a/./N = 0.32. Moreover, in 
subsection 31.3.3, we found the mean of the above sample to be x = 1.11. 
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For the 90% central confidence interval, we require « = 6 = 0.05. From table 30.3, we 
find 


@'(1—«) = © 1(0.95) = 1.65, 
and using (31.31) and (31.32) we obtain 


a_ =X — 1.650% = 1.11 — (1.65)(0.32) = 0.58, 
dy =X + 1.6505 = 1.11 + (1.65)(0.32) = 1.64. 


Thus, the 90% central confidence interval on yp is [0.58, 1.64]. For comparison, the true 
value used to create the sample was p = 1. < 


In the case where the standard error oq in (31.33) is not known in advance, 
one must use a value Ga estimated from the sample. In principle, this complicates 
somewhat the construction of confidence intervals, since properly one should 
consider the two-dimensional joint sampling distribution P (a, Ga|a). Nevertheless, 
in practice, provided Gg is a fairly good estimate of oq the above procedure may 
be applied with reasonable accuracy. In the special case where the sample values 
x; are drawn from a Gaussian distribution with unknown yp and a, it is in fact 
possible to obtain exact confidence intervals on the mean yp, for a sample of any 
size N, using Student’s t-distribution. This is discussed in subsection 31.7.5. 


31.3.6 Estimation of several quantities simultaneously 


Suppose one uses a sample x1,X2,...,xx to calculate the values of several es- 
timators 4, 4@,...,@ (collectively denoted by a) of the quantities a1,a2,...,am 
(collectively denoted by a) that describe the population from which the sample was 
drawn. The joint sampling distribution of these estimators is an M-dimensional 
PDF P(ala) given by 


P(aja)d”@4 = P(x\a) dx. 


> Sample values x;,X2,...,Xy are drawn independently from a Gaussian distribution with 
mean wt and standard deviation o. Suppose we choose the sample mean X and sample stan- 


dard deviation s respectively as estimators jt and G. Find the joint sampling distribution of 
these estimators. 


Since each data value x; in the sample is assumed to be independent of the others, the 
joint probability distribution of sample values is given by 


P(x|y,0) = (2207)? exp 2S" | : 


We may rewrite the sum in the exponent as follows: 


Soi — a? = S01 - 8 +%- wy? 
= S01 — xP + 208 — ») D0 — 3) + SOR wy? 


= Ns’ + N(X— py), 
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where in the last line we have used the fact that }7,(x; — X) = 0. Hence, for given values 
of and o, the sampling distribution is in fact a function only of the sample mean x and 
the standard deviation s. Thus the sampling distribution of x and s must satisfy 


NUG- uP +57] \ av 


53 (31.34) 


P(X, s|u,0) dx ds = (2n07)-®/” exp {- 
where dV = dx; dx2---dxy is an element of volume in the sample space which yields 
simultaneously values of xX and s that lie within the region bounded by [x,x + dx] and 
[s,s + ds]. Thus our only remaining task is to express dV in terms of X and s and their 
differentials. 

Let S be the point in sample space representing the sample (x1, X2,...,xn). For given 
values of X and s, we require the sample values to satisfy both the condition 


‘yy x; = Nx, 
which defines an (N — 1)-dimensional hyperplane in the sample space, and the condition 


(x; — x)? = Ns’, 
oy 


which defines an (N — 1)-dimensional hypersphere. Thus S$ is constrained to lie in the 
intersection of these two hypersurfaces, which is itself an (N —2)-dimensional hypersphere. 
Now, the volume of an (N — 2)-dimensional hypersphere is proportional to s‘~'. It follows 
that the volume dV between two concentric (N — 2)-dimensional hyperspheres of radius 
./Ns and /N(s + ds) and two (N — 1)-dimensional hyperplanes corresponding to x and 
xX + dx is 

dV = As\~ ds dx, 


where A is some constant. Thus, substituting this expression for dV into (31.34), we find 


2 


> *) = P(5|u,0)P(s\o), 
Oo 


N(x — 2 
P(X,s|u,0) = C, exp aS Cys\~* exp (- 
20 (31.35) 
where C; and C, are constants. We have written P(X, s|u,¢0) in this form to show that it 
separates naturally into two parts, one depending only on xX and the other only on s. Thus, 
xX and s are independent variables. Separate normalisations of the two factors in (31.35) 


require 
N \12 N \W-D2 1 
CQ =| and C, =2 : 
(se) - > (a3) T (A(N— 1) 


where the calculation of Cy requires the use of the gamma function, discussed in the 
Appendix. < 


The marginal sampling distribution of any one of the estimators @; is given 
simply by 


Pia) = fos f PGA) day dyad dy 


and the expectation value E[@;] and variance V[4@;] of @; are again given by (31.14) 
and (31.16) respectively. By analogy with the one-dimensional case, the standard 
error og, on the estimator G; is given by the positive square root of V [a]. With 
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several estimators, however, it is usual to quote their full covariance matrix. This 
M x M matrix has elements 


Vig = Cov[ti,2,] = / (a — E[ai])(@) — Elaj))P (ala) da 
= / (a; — Elai))(@; — E[aj])P(xla) ax. 


Fisher’s inequality can be generalised to the multi-dimensional case. Adapting 
the proof given in subsection 31.3.2, one may show that, in the case where the 
estimators are efficient and have zero bias, the elements of the inverse of the 
covariance matrix are given by 


2 
g a (31.36) 


Ty) epee scart 
@ di - | 6a,04; 


where P denotes the population P(x|a) from which the sample is drawn. The 
quantity on the RHS of (31.36) is the element Fj; of the so-called Fisher matrix 
F of the estimators. 


> Calculate the covariance matrix of the estimators X and s in the previous example. 


As shown in (31.35), the joint sampling distribution P(x,s|u,0) factorises, and so the 
estimators X and s are independent. Thus, we conclude immediately that 


Cov[X, s] = 0. 


Since we have already shown in the worked example at the end of subsection 31.3.1 that 
V [x] = 0?/N, it only remains to calculate V[s]. From (31.35), we find 


e 2 Ns? 2 27 LIN —14+ 
BWI=G: | exp ( =) w= (5) et 


where we have evaluated the integral using the definition of the gamma function given in 
the Appendix. Thus, the expectation value of the sample standard deviation is 


Spay GN) 
etl= (5) Tw by” (31.37) 


and its variance is given by 


2 


V[s] = E[s’] — (EIS 


—\N-1 2| EG) | 
P(N — 0) 


We note, in passing, that (31.37) shows that s is a biased estimator of o. < 


The idea of a confidence interval can also be extended to the case where several 
quantities are estimated simultaneously but then the practical construction of an 
interval is considerably more complicated. The general approach is to construct 
an M-dimensional confidence region R in a-space. By analogy with the one- 
dimensional case, for a given confidence level of (say) 1 — «, one first constructs 
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a region Rin a-space, such that 


[ [Pam d@a=1-«a. 


A common choice for such a region is that bounded by the ‘surface’ P(aja) = 
constant. By considering all possible values a and the values of a lying within 
the region R, one can construct a 2M-dimensional region in the combined space 
(a,a). Suppose now that, from our sample x, the values of the estimators are 
Giobs, i = 1,2,...,M. The intersection of the M ‘hyperplanes’ 4; = @jops with 
the 2M-dimensional region will determine an M-dimensional region which, when 
projected onto a-space, will determine a confidence limit R at the confidence 
level 1 — «. It is usually the case that this confidence region has to be evaluated 
numerically. 

The above procedure is clearly rather complicated in general and a simpler 
approximate method that uses the likelihood function is discussed in subsec- 
tion 31.5.5. As a consequence of the central limit theorem, however, in the 
large-sample limit, N — 00, the joint sampling distribution P(alja) will tend, in 
general, towards the multivariate Gaussian 


P(ala) 


3 1 A 

(2n)M2]y|I72 exp [ 7O(a, a)| , (31.38) 
where V is the covariance matrix of the estimators and the quadratic form Q is 
given by 


Q(4,a) = (A—a)'V-'(4—a). 


Moreover, in the limit of large N, the inverse covariance matrix tends to the 
Fisher matrix F given in (31.36), ie. V-! > F. 

For the Gaussian sampling distribution (31.38), the process of obtaining confi- 
dence intervals is greatly simplified. The surfaces of constant P(a|a) correspond 
to surfaces of constant Q(4,a), which have the shape of M-dimensional ellipsoids 
in 4-space, centred on the true values a. In particular, let us suppose that the 
ellipsoid Q(a,a) = c (where c is some constant) contains a fraction 1 — « of the 
total probability. Now suppose that, from our sample x, we obtain the values Aops 
for our estimators. Because of the obvious symmetry of the quadratic form Q 
with respect to a and 4, it is clear that the ellipsoid Q(a,4,;) = c in a-space that 
is centred on 4,,; should contain the true values a with probability 1 — «. Thus 
Q(a, Aops) = c defines our required confidence region R at this confidence level. 
This is illustrated in figure 31.4 for the two-dimensional case. 

It remains only to determine the constant c corresponding to the confidence 
level 1 — « As discussed in subsection 30.15.2, the quantity Q(a,a) is distributed 
as a y* variable of order M. Thus, the confidence region corresponding to the 
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a ay 


(a) (b) 


~ ay - ay 


Figure 31.4 (a) The ellipse Q(4,a) = c in 4-space. (b) The ellipse Q(a, Agus) = c 
in a-space that corresponds to a confidence region R at the level 1 — «, when 
c satisfies (31.39). 


confidence level 1 — « is given by Q(a,4 ps) = c, where the constant c satisfies 
[ Pender =1-4, (31.39) 
0 


and P(73,) is the chi-squared PDF of order M, discussed in subsection 30.9.4. This 
integral may be evaluated numerically to determine the constant c. Alternatively, 
some reference books tabulate the values of c corresponding to given confidence 
levels and various values of M. A representative selection of values of c is given 
in table 31.2; there the number of degrees of freedom is denoted by the more 
usual n, rather than M. 


31.4 Some basic estimators 


In many cases, one does not know the functional form of the population from 
which a sample is drawn. Nevertheless, in a case where the sample values 
X1,X2,...,Xn are each drawn independently from a one-dimensional population 
P(x), it is possible to construct some basic estimators for the moments and central 
moments of P(x). In this section, we investigate the estimating properties of the 
common sample statistics presented in section 31.2. In fact, expectation values 
and variances of these sample statistics can be calculated without prior knowledge 
of the functional form of the population; they depend only on the sample size N 
and certain moments and central moments of P(x). 


31.4.1 Population mean 


Let us suppose that the parent population P(x) has mean p and variance o?. An 
obvious estimator jt of the population mean is the sample mean x. Provided yu 
and o? are both finite, we may apply the central limit theorem directly to obtain 
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10 5 2:5 1 0.5 0.1 

1.5710-4 3.93 10-3 2.71 3.84 5.02 6.63 7.88 10.83 
2.01 107? 0.103 4.61 5.99 7.38 9.21 10.60 13.81 
0.115 0.352 6.25 7.81 9.35 11.34 12.84 16.27 
0.297 0.711 7.78 949 11.14 13.28 1486 18.47 
0.554 1.15 9.24 11.07 12.83 15.09 16.75 20.52 
0.872 1.64 |} 10.64 12.59 1445 1681 1855 22.46 
1.24 2.17 |} 12.02 1407 1601 1848 20.28 24.32 
1.65 2.73 |) 13.36 15.51 17.53 20.09 21.95 26.12 
2.09 3.33 |} 14.68 16.92 19.02 21.67 23.59 27.88 
2.56 3.94 }} 15.99 18.31 20.48 23.21 25.19 29.59 
3.05 4.57 |] 17.28 19.68 21.92 24.73 26.76 31.26 
3.57 5.23 |} 18.55 21.03 23.34 26.22 28.30 32.91 
4.11 5.89 |} 19.81 22.36 24.74 27.69 29.82 34.53 
4.66 6.57 |) 21.06 23.68 26.12 29.14 31.32 36.12 
5.23 7.26 |) 22.31 25.00 27.49 30.58 32.80 37.70 
5.81 7.96 |) 23.54 26.30 28.85 32.00 34.27 39.25 
6.41 8.67 |] 24.77 27.59 30.19 33.41 35.72 40.79 
7.01 9.39 |) 25.99 28.87 31.53 3481 37.16 42.31 
7.63 10.12 |] 27.20 30.14 32.85 36.19 38.58 43.82 
8.26 10.85 |] 28.41 31.41 34.17 37.57 40.00 45.31 
8.90 11.59 |] 29.62 32.67 35.48 38.93 4140 46.80 
9.54 12.34 |] 30.81 33.92 36.78 40.29 42.80 48.27 
10.20 13.09 || 32.01 35.17 38.08 41.64 44.18 49.73 
10.86 13.85 || 33.20 36.42 39.36 42.98 45.56 51.18 
11.52 14.61 || 34.38 37.65 40.65 4431 46.93 52.62 
14.95 18.49 || 40.26 43.77 46.98 50.89 53.67 59.70 
22.16 26.51 }[ 51.81 55.76 59.34 63.69 66.77 73.40 
29.71 34.76 || 63.17 67.50 71.42 76.15 79.49 86.66 
37.48 43.19 || 74.40 79.08 83.30 88.38 91.95 99.61 
45.44 51.74 || 85.53. 90.53 95.02 1004 104.2 112.3 
53.54 60.39 |] 96.58 101.9 1066 1123 1163 1248 
61.75 69.13 |) 107.6 113.1 118.1 1241 1283 137.2 
70.06 77.93 || 118.5 124.3 129.6 135.8 140.2 149.4 


Table 31.2 The tabulated values are those which a variable distributed as 7? 
with n degrees of freedom exceeds with the given percentage probability. For 
example, a variable having a 7? distribution with 14 degrees of freedom takes 
values in excess of 21.06 on 10% of occasions. 
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exact expressions, valid for samples of any size N, for the expectation value and 
variance of x. From parts (i) and (ii) of the central limit theorem, discussed in 
section 30.10, we immediately obtain 


E[x] =m VERS. (31.40) 


Thus we see that X is an unbiased estimator of uw. Moreover, we note that the 
standard error in x is ¢/./N, and so the sampling distribution of x becomes more 
tightly centred around yu as the sample size N increases. Indeed, since V[x] > 0 
as N — 00, X is also a consistent estimator of uu. 

In the limit of large N, we may in fact obtain an approximate form for the 
full sampling distribution of x. Part (ili) of the central limit theorem (see section 
30.10) tells us immediately that, for large N, the sampling distribution of x is 
given approximately by the Gaussian form 


P(X|i,0) 


exp | oe ; 
2no2/N 207/N 
Note that this does not depend on the form of the original parent population. 
If, however, the parent population is in fact Gaussian then this result is exact 
for samples of any size N (as is immediately apparent from our discussion of 
multiple Gaussian distributions in subsection 30.9.1). 


31.4.2 Population variance o* 


An estimator for the population variance o? is not so straightforward to define 


as one for the mean. Complications arise because, in many cases, the true mean 
of the population py is not known. Nevertheless, let us begin by considering the 
case where in fact 4 is known. In this event, a useful estimator is 


be el 1a 
Dies } Dee a eee) 
0 N im (x; — H) (x > “) Ww. (31.41) 


> Show that o? is an unbiased and consistent estimator of the population variance o?. 


The expectation value of G2 is given by 


from which we see that the estimator is unbiased. The variance of the estimator is 


N 
V[o2] = a p> x? 
i=1 


in which we have used that fact that V[u?] =0 and V[x?] = E[x4] — (E[x?7])? = wa — 16, 


1 1 
+ VW] = VL] = Flea — 10), 
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where yu, is the rth population moment. Since 62 is unbiased and V[o?] -0asN> om, 
showing that it is also a consistent estimator of o”, the result is established. < 


If the true mean of the population is unknown, however, a natural alternative 
is to replace yu by x in (31.41), so that our estimator is simply the sample variance 
s* given by 


oe eae 
i=l i=1 

In order to determine the properties of this estimator, we must calculate E[s7] 
and V[s*]. This task is straightforward but lengthy. However, for the investigation 
of the properties of a central moment of the sample, there exists a useful trick 
that simplifies the calculation. We can assume, with no loss of generality, that 
the mean 4, of the population from which the sample is drawn is equal to zero. 
With this assumption, the population central moments, v,, are identical to the 
corresponding moments y,, and we may perform our calculation in terms of the 
latter. At the end, however, we replace , by v, in the final result and so obtain 
a general expression that is valid even in cases where 1; # 0. 


Calculate E[s*] and V[s?] for a sample of size N. 


The expectation value of the sample variance s* for a sample of size N is given by 


= NEL] — yp Do + Dy ; (31.42) 


The number of terms in the double summation in (31.42) is N(N — 1), so we find 


1 
E[s?] = E[x?] — qi NE [x7] + NIN — L)E[x:x;]). 
Now, since the sample elements x; and x; are independent, E[x;x;] = E[xiJE[x;] = 9, 
assuming the mean j; of the parent population to be zero. Denoting the rth moment of 
the population by y,, we thus obtain 

In N-1 N-1, 
= —_ 43 
N WH Wo: (31.43) 


where in the last line we have used the fact that the population mean is zero, and so 
Hy = V7 = o*. However, the final result is also valid in the case where p, # 0. 

Using the above method, we can also find the variance of s?, although the algebra is 
rather heavy going. The variance of s? is given by 


V[s?] = E[s*] —(E[s]), (31.44) 


E [s7] M2 


where E[s?] is given by (31.43). We therefore need only consider how to calculate E[s*], 
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where s* is given by 


Oey? Oo? Hos (xi 
N? 2 ae Nt (31.45) 
We will consider in turn each of the three terms on the RHS. In the first term, the sum 
(>°, x7)? can be written as 


(xx) = Soa t Sox}, 


where the first sum contains N terms and the second contains N(N — 1) terms. Since the 
sample elements x; and x; are assumed independent, we have E[x;xj] = E[x;]E[xj] = 16, 
and so 


2, 
(= «) = Nyy + N(N — 16. 


Turning to the second term on the RHS of (31.45), 
2 
(= «) (x “) = > xt + i X}X; + a X}XF + 2 XPXjXk- 
i i i ij ij 
i#i ii 


Since the mean of the population has been assumed to equal zero, te expectation values 
of the second and fourth sums on the RHS vanish. The first and third sums contain N 
and N(N — 1) terms respectively, and so 


2 
(= “) i s) = Nyy + N(N — 1)6. 


Finally, we consider the third term on the RHS of (31.45), and write 


(xs) = 2% +208 ae ee + > XPXjXe + yy XiXjXKXI. 
: iu jai 


ik ijkl 
heii pk iti 


The expectation values of the second, fourth and fifth sums are zero, and the first and third 
sums contain N and 3N(N-—1) terms respectively (for the third sum, there are N(N — 1)/2 
ways of choosing i and j, and the multinomial coefficient of XPX} is 4!/(2!2!) = 6). Thus 


4 
& “) = Nyy +3N(N — Lyd 


Collecting together terms, we therefore obtain 
(N—1) (N — 1)(N?—2N +3) 5 
N? ba + N3 Hy, 
which, together with the result (31.43), may be substituted into (31.44) to obtain finally 
(N — 1)? (N — 1)(N — 3) 
V[s] = Pe 2 = ee 
= = [(N — va — (N — 3)v3], (31.47) 


E[s*] (31.46) 
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where in the last line we have used again the fact that, since the population mean is zero, 
Ly = v,. However, result (31.47) holds even when the population mean is not zero. < 


From (31.43), we see that s? is a biased estimator of o*, although the bias 
becomes negligible for large N. However, it immediately follows that an unbiased 
estimator of o is given simply by 

peg 

N-1 

where the multiplicative factor N/(N —1) is often called Bessel’s correction. Thus 

in terms of the sample values x;, i = 1,2,...,N, an unbiased estimator of the 
population variance o is given by 


(31.48) 


N 
= — Si — x). (31.49) 
i=1 


Using (31.47), we find that the variance of the estimator @2 is 


4 N \? 1 N-3 
ces (qa) Bar (« ‘ a1") 


where v, is the rth central moment of the parent population. We note that, 
since E[o2] = o* and V[o2] — 0 as N > «, the statistic o2 is also a consistent 
estimator of the population variance. 


31.4.3 Population standard deviation o 


The standard deviation o of a population is defined as the positive square root of 
the population variance o? (as, indeed, our notation suggests). Thus, it is common 
practice to take the positive square root of the variance estimator as our estimator 
for o. Thus, we take 


eS (0?) — (31.50) 


where o? is given by either (31.41) or (31.48), depending on whether the population 
mean y is known or unknown. Because of the square root in the definition of 
G, it is not possible in either case to obtain an exact expression for E[é] and 
V[G]. Indeed, although in each case the estimator is the positive square root of 
an unbiased estimator of o, it is not itself an unbiased estimator of o. However, 
the bias does becomes negligible for large N. 


> Obtain approximate expressions for E[Gé] and V[é] for a sample of size N in the case 
where the population mean p is unknown. 


As the population mean is unknown, we use (31.50) and (31.48) to write our estimator in 
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the form 


N 1/2 
= (74) : 


where s is the sample standard deviation. The expectation value of this estimator is given 


by 
1/2 1/2 
Hels (qo) EU(s’)'7] = (a) (Els)? =. 


An approximate expression for the variance of ¢ may be found using (31.47) and is given 


by 


Viel = VII & 


Te V [s7] 
=E[s?] 


i. E ae 


V [s?] respectively, we obtain 


and 
= 1325 
N-1” 


Using the expressions (31.43) and (31.47) for E[s? 


" 1 
Klele 4Nv> (».- 


r= Last 
rile 


31.4.4 Population moments 1, 


We may straightforwardly generalise our discussion of estimation of the popula- 
tion mean “(= py) in subsection 31.4.1 to the estimation of the rth population 
moment y,. An obvious choice of estimator is the rth sample moment m,. The 
expectation value of m, is given by 


E[m,] = => ebs 


and so it is an unbiased estimator of 11,. 
The variance of m, may be found in a similar manner, although the calculation 
is a little more complicated. We find that 


= Ur, 


V [m,] = E[(oy — by ial 


2 
1 
= wee (san) 
i 


= aE a x + xx ak 


i j#i 
1 
= yl — + ae DD ERX (31.51) 
i ffi 


1249 


STATISTICS 


However, since the sample values x; are assumed to be independent, we have 


E[x}xj] = ELGJERS] = a7. (31.52) 


The number of terms in the sum on the RHS of (31.51) is N(N—1), and so we find 


N45 pop =e 

we N 
Since E[m,] = py, and V[m,] > 0 as N — oo, the rth sample moment m, is also 
a consistent estimator of pU;. 


» Find the covariance of the sample moments m, and m, for a sample of size N. 


We obtain the covariance of the sample moments m, and m, in a similar manner to that 
used above to obtain the variance of m,. From the definition of covariance, we have 


1 
Vim] = Sir — ue (31.53) 


Cov[m,,m;] = E[(m, = Lr )(ms me Hs)] 


= awk (x: w— Ni) (sina) 


1 r S S ie 
= yk | DI + DD xix — Nie DE — Nas Dx + NPhs 
i i j#i j i 
Assuming the x; to be independent, we may again use result (31.52) to obtain 
1 
Cov[im,,ms] = [Nits + N(N — Uperts — N Lp bs — N? sly +N? UUs] 

il i: N-1 es 

a Wires N bs — Hr Hs 

_ Lr+s — Urbs 

= ogee 

We note that by setting r = s, we recover the expression (31.53) for V[m,]. < 


31.4.5 Population central moments v, 


We may generalise the discussion of estimators for the second central moment v2 
(or equivalently o?) given in subsection 31.4.2 to the estimation of the rth central 
moment v,. In particular, we saw in that subsection that our choice of estimator 
for v2 depended on whether the population mean 1; is known; the same is true 
for the estimation of v,. 

Let us first consider the case in which 1; is known. From (30.54), we may write 
Vv, as 

Ve = te — Capt an to + (OL Cea icedt + oo + (DCG — Da. 

If y; is known, a suitable estimator is obviously 


$, =m, —"Cymp apr +++ + (1) "Cem nuh Fe FED TOG — Dut 


where m, is the rth sample moment. Since 4, and the binomial coefficients are 
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(known) constants, it is immediately clear that E[},] = v,, and so ?, is an unbiased 
estimator of v,. It is also possible to obtain an expression for V[,], though the 
calculation is somewhat lengthy. 

In the case where the population mean py; is not known, the situation is more 
complicated. We saw in subsection 31.4.2 that the second sample moment nz (or 
s*) is not an unbiased estimator of v2 (or o7). Similarly, the rth central moment of 
a sample, n,, is not an unbiased estimator of the rth population central moment 
v,. However, in all cases the bias becomes negligible in the limit of large N. 

As we also found in the same subsection, there are complications in calculating 
the expectation and variance of nz; these complications increase considerably for 
general r. Nevertheless, we have derived already in this chapter exact expressions 
for the expectation value of the first few sample central moments, which are valid 
for samples of any size N. From (31.40), (31.43) and (31.46), we find 


E[n] = 0, 
N-1 
E[m] = "2 (31.54) 
E[n3] [(N — 1)vq + (N? — 2N + 3)V3] 
By similar arguments it can be shown that 
N—1\(N—2 
E[n3] = Wen), (31.55) 
N-1._., , 
E [ng] = wi [(N* — 3N + 3)v4 + 3(2N — 3)v5]. (31.56) 
From (31.54) and (31.55), we see that unbiased estimators of v2 and v3 are 
N 
t> = 31.57 
= ym ( ) 
N2 
a 31.58 
= WonWwoay” ( ) 


where (31.57) simply re-establishes our earlier result that e= Ns?/(N — 1) is an 
unbiased estimator of o?. 

Unfortunately, the pattern that appears to be emerging in (31.57) and (31.58) 
is not continued for higher r, as is seen immediately from (31.56). Nevertheless, 
in the limit of large N, the bias becomes negligible, and often one simply takes 
), =n,. For large N, it may be shown that 


E[n,] © v, 


1 
V [n,] © wr _ v2 + rvove 4 — 2rv,—1Vr+1) 


1 
N 


Cov[n,, ns] = (Vr+s = VeVs 1 LSV2Vp—1Vs—1 — TVp—-1 Vs HL — SVs—1Vr4 1) 
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31.4.6 Population covariance Cov[x, y] and correlation Corrt[x, y] 


So far we have assumed that each of our N independent samples consists of 
a single number x;. Let us now extend our discussion to a situation in which 
each sample consists of two numbers x;,y;, which we may consider as being 
drawn randomly from a two-dimensional population P(x, y). In particular, we 
now consider estimators for the population covariance Cov[x, y] and for the 
correlation Corr[x, y]. 

When px and py are known, an appropriate estimator of the population covari- 
ance is 


N 

— 1 

Cov[x, y] =X) — Uxby = (5 SL vs] — HsHly. (31.59) 
i=1 


This estimator is unbiased since 


es 1 N 
E [Covlx,y]] = SE > wy pty = Elxivil — petty = Covls, y] 
i=1 


Alternatively, if yu. and py are unknown, it is natural to replace p, and py in 
(31.59) by the sample means x and j respectively, in which case we recover the 
sample covariance V,, = Xy — xy discussed in subsection 31.2.4. This estimator 
is biased but an unbiased estimator of the population covariance is obtained by 
forming 


Cov[x, y] = aN. (31.60) 


> Calculate the expectation value of the sample covariance Vy, for a sample of size N. 


The sample covariance is given by 


--(:5)-() (6) 


Thus its expectation value is given by 


eles le) 
Elxiy] — pe ae 
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Since the number of terms in the double sum on the RHS is N(N — 1), we have 


1 
E(Vxy] = E[xiyi] — rE [xiyi] + N(N — 1DE[xiy{]) 


1 
= E[xiyi] — rine [xiv] + N(N — 1)E [xiJE[y)]) 


N 


= Elxy] — ~ (Ebey] + (NV — 1s) = S*covpx yl, 
N 


where we have used the fact that, since the samples are independent, E[x;y;] = E[x;]E[y;]. < 


It is possible to obtain expressions for the variances of the estimators (31.59) 
and (31.60) but these quantities depend upon higher moments of the population 
P(x,y) and are extremely lengthy to calculate. 

Whether the means yx, and my, are known or unknown, an estimator of the 
population correlation Corr[x, y] is given by 


see. Covl x, y 
Cort[x, y] = el (31.61) 
xOy 


where Cov[x, y], G. and Gy are the appropriate estimators of the population co- 
variance and standard deviations. Although this estimator is only asymptotically 
unbiased, i.e. for large N, it is widely used because of its simplicity. Once again 
the variance of the estimator depends on the higher moments of P(x, y) and is 
difficult to calculate. 
In the case in which the means pi, and py are unknown, a suitable (but biased) 
estimator is 
N Vyy N 


Conlayl = N—is,s, N—-1” 
Sy Sy 


(31.62) 


where s, and s, are the sample standard deviations of the x; and y; respectively 
and r,, is the sample correlation. In the special case when the parent population 
P(x, y) is Gaussian, it may be shown that, if o = Corr[x, y], 


p(l — p?) 
2N 
Vir] = =(l — py +O(N~), (31.64) 


E[rsxy] = p— + O(N~), (31.63) 


from which the expectation value and variance of the estimator Corr[x, y] may 
be found immediately. 

We note finally that our discussion may be extended, without significant al- 
teration, to the general case in which each data item consists of n numbers 


Xis Vis «+ +5 Zi 
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31.4.7 A worked example 


To conclude our discussion of basic estimators, we reconsider the set of experi- 
mental data given in subsection 31.2.4. We carry the analysis as far as calculating 
the standard errors in the estimated population parameters, including the popu- 
lation correlation. 


>» Ten UK citizens are selected at random and their heights and weights are found to be as 
follows (to the nearest cm or kg respectively): 


Person Any 0B) Ce EDs WcEio gEs) Ge GHA wal) ey 
Height (cm) 194 168 177 180 171 190 151 169 175 182 
Weight (kg) 75 53 72 80 75 75 57 67 46 68 


Estimate the means, px, and py, and standard deviations, o, and oy, of the two-dimensional 
joint population from which the sample was drawn, quoting the standard error on the esti- 
mate in each case. Estimate also the correlation Corr[x, y] of the population, and quote the 
standard error on the estimate under the assumption that the population is a multivariate 
Gaussian. 


In subsection 31.2.4, we calculated various sample statistics for these data. In particular, 
we found that for our sample of size N = 10, 


X=175.7, f=668, 


$= 11:6, sy = 10.6, Ixy = 0.54. 

Let us begin by estimating the means jw, and my. As discussed in subsection 31.4.1, the 
sample mean is an unbiased, consistent estimator of the population mean. Moreover, the 
standard error on X (say) is ox/JN. In this case, however, we do not know the true value 
of o, and we must estimate it using ¢, = JN/(N —1)s,. Thus, our estimates of uw. and 
Hy, with associated standard errors, are 


fp = 3+ = 175.7439, 
N—1 


A s Sy 
MW =YX JN =r 


We now turn to estimating o, and o,. As just mentioned, our estimate of o, (say) 
is 6, = JN/(N — Dox. Its variance (see the final line of subsection 31.4.3) is given 


approximately by 
1 N—3 
V [6] » —— [ v4 — v3 } . 
9] ~ any (>. N— v2) 


Since we do not know the true values of the population central moments v2 and v4, we 
must use their estimated values in this expression. We may take }, = o2 = (4)°, which we 
have already calculated. It still remains, however, to estimate v4. As implied near the end 
of subsection 31.4.5, it is acceptable to take $4 = ny. Thus for the x; and y; values, we have 


= 66.8 + 3.5. 


(x; — x)* = 53.411.6 


=> 
Rs 
= 
z 
ll 
Zlr 
Me 


P {> 7 
Ga)y = 5 Lo —j)* = 277325 
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Substituting these values into (31.50), we obtain 


é,= (4) - sy + (V[éJ)'? = 122467, (31.65) 
N-1 a =. 
Gy = (a7) me Sy (V[6,))\/? = 11.2436. (31.66) 
: N-1 
Finally, we estimate the population correlation Corr[x, y], which we shall denote by p. 


From (31.62), we have 

N 
N-1 
Under the assumption that the sample was drawn from a two-dimensional Gaussian 
population P(x, y), the variance of our estimator is given by (31.64). Since we do not know 
the true value of p, we must use our estimate p. Thus, we find that the standard error Ap 
in our estimate is given approximately by 


10/1 es 
Ap @ [1 — (0.60)2}? = 0.05. « 


p= yy = 0.60. 


31.5 Maximum-likelihood method 


The population from which the sample x1,x2,...,xy is drawn is, in general, 
unknown. In the previous section, we assumed that the sample values were inde- 
pendent and drawn from a one-dimensional population P(x), and we considered 
basic estimators of the moments and central moments of P(x). We did not, how- 
ever, assume a particular functional form for P(x). We now discuss the process 
of data modelling, in which a specific form is assumed for the population. 

In the most general case, it will not be known whether the sample values are 
independent, and so let us consider the full joint population P(x), where x is the 
point in the N-dimensional data space with coordinates x1,x2,...,xv. We then 
adopt the hypothesis H that the probability distribution of the sample values has 
some particular functional form L(x;a), dependent on the values of some set of 
parameters a;, i = 1,2,...,m. Thus, we have 


P(x\a, H) = L(x;a), 


where we make explicit the conditioning on both the assumed functional form and 
on the parameter values. L(x;a) is called the likelihood function. Hypotheses of this 
type form the basis of data modelling and parameter estimation. One proposes a 
particular model for the underlying population and then attempts to estimate from 
the sample values x1, X2,...,xy the values of the parameters a defining this model. 


>A company measures the duration (in minutes) of the N intervals x;, i = 1,2,...,N 
between successive telephone calls received by its switchboard. Suppose that the sample 


values x; are drawn independently from the distribution P(x|\t) = (1/t) exp(—x/t), where t 
is the mean interval between calls. Calculate the likelihood function L(x;t). 


Since the sample values are independent and drawn from the stated distribution, the 
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Figure 31.5 Examples of the likelihood function (31.67) for samples of dif- 
ferent size N. In each case, the true value of the parameter is t = 4 and the 
sample values x; are indicated by the short vertical lines. For the purposes 
of illustration, in each case the likelihood function is normalised so that its 
maximum value is unity. 
likelihood is given by 
L(x; 1) = P(xi|t)P (xalt) - +» P(x) 
1 x,\ 1 x2 1 XN 
leg la) a) 
1 1 
iN exp mac +XxXofeeet x9)| : (31.67) 


which is to be considered as a function of t, given that the sample values x; are fixed. < 
The likelihood function (31.67) depends on just a single parameter t. Plots of 


the likelihood function, considered as a function of t, are shown in figure 31.5 for 
samples of different size N. The true value of the parameter t used to generate the 
sample values was 4. In each case, the sample values x; are indicated by the short 
vertical lines. For the purposes of illustration, the likelihood function in each 
case has been scaled so that its maximum value is unity (this is, in fact, common 
practice). We see that when the sample size is small, the likelihood function is 
very broad. As N increases, however, the function becomes narrower (its width is 
inversely proportional to \/N) and tends to a Gaussian-like shape, with its peak 
centred on 4, the true value of t. We discuss these properties of the likelihood 
function in more detail in subsection 31.5.6. 
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L(x;a) L(x; a) 
‘ =a a 
a 
L(x;a) 
| (c) 
7 =a es a 
a a 


Figure 31.6 Typical shapes of one-dimensional likelihood functions L(x; a) 
encountered in practice, when, for illustration purposes, it is assumed that the 
parameter a is restricted to the range zero to infinity. The ML estimator in 
the various cases occurs at: (a) the only stationary point; (b) one of several 
stationary points; (c) an end-point of the allowed parameter range that is not 
a stationary point (although stationary points do exist); (d) an end-point of 
the allowed parameter range in which no stationary point exists. 


31.5.1 The maximum-likelihood estimator 


Since the likelihood function L(x;a) gives the probability density associated with 
any particular set of values of the parameters a, our best estimate 4 of these 
parameters is given by the values of a for which L(x;a) is a maximum. This is 
called the maximum-likelihood estimator (or ML estimator). 

In general, the likelihood function can have a complicated shape when con- 
sidered as a function of a, particularly when the dimensionality of the space of 
parameters a,d2,...,am is large. It may be that the values of some parameters 
are either known or assumed in advance, in which case the effective dimension- 
ality of the likelihood function is reduced accordingly. However, even when the 
likelihood depends on just a single parameter a (either intrinsically or as the 
result of assuming particular values for the remaining parameters), its form may 
be complicated when the sample size N is small. Frequently occurring shapes of 
one-dimensional likelihood functions are illustrated in figure 31.6, where we have 
assumed, for definiteness, that the allowed range of the parameter a is zero to 
infinity. In each case, the ML estimate @ is also indicated. Of course, the ‘shape’ of 
higher-dimensional likelihood functions may be considerably more complicated. 

In many simple cases, however, the likelihood function L(x;a) has a single 
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maximum that occurs at a stationary point (the likelihood function is then termed 
unimodal). In this case, the ML estimators of the parameters a;, i = 1,2,...,M, 
may be found without evaluating the full likelihood function L(x;a). Instead, one 
simply solves the M simultaneous equations 


es 
a =0 for i = 1,2,...,M. (31.68) 

Oi |a—4 
Since Inz is a monotonically increasing function of z (and therefore has the 
same stationary points), it is often more convenient, in fact, to maximise the 
log-likelihood function, \n L(x;a), with respect to the a;. Thus, one may, as an 
alternative, solve the equations 


olnL 


Oa; 
0a; aon 


=0  fori=1,2,...,M. (31.69) 


Clearly, (31.68) and (31.69) will lead to the same ML estimates 4 of the parameters. 
In either case, it is, of course, prudent to check that the point a = 4 is a local 
maximum. 


> Find the ML estimate of the parameter t in the previous example, in terms of the measured 
bales k= 1 


From (31.67), the log-likelihood function in this case is given by 


N 


al 1 Xj /T Xi 
In L(x:1) dom (Xe ) > (t+). (31.70) 


i=1 


Differentiating with respect to the parameter t and setting the result equal to zero, we find 


dlnL “fl x; 
Ot > (3 *) 0 


i=1 


Thus the ML estimate of the parameter t is given by 


, 1 
t=5 2% (31.71) 


which is simply the sample mean of the N measured intervals. < 


In the previous example we assumed that the sample values x; were drawn 
independently from the same parent distribution. The ML method is more flexible 
than this restriction might seem to imply, and it can equally well be applied to 
the common case in which the samples x; are independent but each is drawn 
from a different distribution. 


1258 


31.5 MAXIMUM-LIKELIHOOD METHOD 


pIn an experiment, N independent measurements x; of some quantity are made. Suppose 
that the random measurement error on the ith sample value is Gaussian distributed with 
mean zero and known standard deviation o;. Calculate the ML estimate of the true value 
Lt of the quantity being measured. 


As the measurements are independent, the likelihood factorises: 
e 
L(x; w, {oe}) = [] Pele, 0”), 
i=1 


where {o;,} denotes collectively the set of known standard deviations o1,02,...,¢y. The 
individual distributions are given by 


P(xi|u, 07) nk | : 


1 
2207 a | 207 


and so the full log-likelihood function is given by 
N 
, 1 2 (xi = hy? 
In L(x; 1, {ox }) = =5 a, none; y+ ee ; 
Differentiating this expression with respect to w and setting the result equal to zero, we 


find 
N 


élnL i- 
Ly Se =i 0, 
Ou 0; 


i=l 


from which we obtain the ML estimator 


N 2 

a 1 (Xi/ 0; 
p= Veriler) i] (31.72) 

Vi-1 1/97) 
This estimator is commonly used when averaging data with different statistical weights 
w; = 1/7. We note that when all the variances cg? have the same value the estimator 


reduces to the sample mean of the data x;. < 


There is, in fact, no requirement in the ML method that the sample values 
be independent. As an illustration, we shall generalise the above example to a 
case in which the measurements x; are not all independent. This would occur, for 
example, if these measurements were based at least in part on the same data. 


>In an experiment N measurements x; of some quantity are made. Suppose that the random 
measurement errors on the samples are drawn from a joint Gaussian distribution with mean 


zero and known covariance matrix V. Calculate the ML estimate of the true value q of the 
quantity being measured. 


From (30.148), the likelihood in this case is given by 
1 as. 
L(x; 4,V) = Qrryz XP [-3(x = pAIV A(X u1)| > 


where x is the column matrix with components x1, X2,...,x\y and 1 is the column matrix 
with all components equal to unity. Thus, the log-likelihood function is given by 


In L(x; 4, V) = —3 [N In(2z) + In|V| + (x = wA)TV (x = 1]. 


1259 


STATISTICS 


Differentiating with respect to w and setting the result equal to zero gives 
Ink _ 

Ou 

Thus, the ML estimator is given by 
uh dae -_ iV Dix 
vr yO , 


In the case of uncorrelated errors in measurement, (VV); = 6;;/07 and our estimator 
reduces to that given in (31.72). < 


= 1'v-(x — 1) =0. 


he 


In all the examples considered so far, the likelihood function has been effectively 
one-dimensional, either instrinsically or under the assumption that the values of 
all but one of the parameters are known in advance. As the following example 
involving two parameters shows, the application of the ML method to the 
estimation of several parameters simultaneously is straightforward. 


> In an experiment N measurements x; of some quantity are made. Suppose the random error 
on each sample value is drawn independently from a Gaussian distribution of mean zero but 
unknown standard deviation o (which is the same for each measurement ). Calculate the ML 
estimates of the true value qt of the quantity being measured and the standard deviation o 
of the random errors. 


In this case the log-likelihood function is given by 


in 2 (xj — uw)? 
InL(x;p,0)=—5 >) In(2z0 a aa 
i=l 


Taking partial derivatives of In L with respect to uw and o and setting the results equal to 
zero at the joint estimate fi,é, we obtain 


Ng a 
y= = 6 (31.73) 


N Ge N 1 
ie = Dee (31.74) 


i=l i=1 


In principle, one should solve these two equations simultaneously for ji and G, but in this 
case we notice that the first is solved immediately by 


1x 
= ye 


where x is the sample mean. Substituting this result into the second equation, we find 


where s is the sample standard deviation. As shown in subsection 31.4.3, s is a biased 
estimator of o. The reason why the ML method may produce a biased estimator is 
discussed in the next subsection. < 
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31.5.2 Transformation invariance and bias of ML estimators 


An extremely useful property of ML estimators is that they are invariant to 
parameter transformations. Suppose that, instead of estimating some parameter 
a of the assumed population, we wish to estimate some function «(a) of the 
parameter. The ML estimator &(a) is given by the value assumed by the function 
a(a) at the maximum point of the likelihood, which is simply equal to «(@). Thus, 
we have the very convenient property 


&(a) = a(A). 


We do not have to worry about the distinction between estimating a and estimat- 
ing a function of a. This is not true, in general, for other estimation procedures. 


>A company measures the duration (in minutes) of the N intervals x;, i = 1,2,...,N, 
between successive telephone calls received by its switchboard. Suppose that the sample 


values x; are drawn independently from the distribution P(x|t) = (1/t) exp(—x/t). Find the 
ML estimate of the parameter 4 = 1/t. 


This is the same problem as the first one considered in subsection 31.5.1. In terms of the 
new parameter 4, the log-likelihood function is given by 


N 


N 
In L(x;2) = So Ine) = $7 (Ind = Ax). 
i=1 


i=1 


Differentiating with respect to 4 and setting the result equal to zero, we have 


dnt YW /1 
a (Gj xi) 0. 


i=1 


Thus, the ML estimator of the parameter 2 is given by 


N -1 
= ($d) =x, (31.75) 


Referring back to (31.71), we see that, as expected, the ML estimators of 4 and t are 
related by A= 1/7. < 


Although this invariance property is useful it also means that, in general, ML 
estimators may be biased. In particular, one must be aware of the fact that even 
if @ is an unbiased ML estimator of a it does not follow that the estimator &(a) is 
also unbiased. In the limit of large N, however, the bias of ML estimators always 
tends to zero. As an illustration, it is straightforward to show (see exercise 31.8) 
that the ML estimators ¢ and / in the above example have expectation values 


N 
N-1 
In fact, since t = X and the sample values are independent, the first result follows 


immediately from (31.40). Thus, t is unbiased, but d= 1/% is biased, albeit that 
the bias tends to zero for large N. 


Eft]=t and = Ef= Z. (31.76) 
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31.5.3 Efficiency of ML estimators 


We showed in subsection 31.3.2 that Fisher’s inequality puts a lower limit on the 
variance V[@] of any estimator of the parameter a. Under our hypothesis H on 
p. 1255, the functional form of the population is given by the likelihood function, 
ie. P(x|a,H) = L(x;a). Thus, if this hypothesis is correct, we may replace P by 
L in Fisher’s inequality (31.18), which then reads 


ri ab\? a InL 
> 
rial (: 9 xa) fe | da? , 


where b is the bias in the estimator a. We usually denote the RHS by Vmin. 

An important property of ML estimators is that if there exists an efficient 
estimator deg, i.e. one for which V [Ger] = Vin, then it must be the ML estimator 
or some function thereof. This is easily shown by replacing P by L in the proof 
of Fisher’s inequality given in subsection 31.3.2. In particular, we note that the 
equality in (31.22) holds only if h(x) = cg(x), where c is a constant. Thus, if an 
efficient estimator dey exists, this is equivalent to demanding that 

OlnL 
da 


= clon — a(a)]. 


Now, the ML estimator Gm is given by 


olnL 
Oa 


=0 = C[der — e(4m)] = 9, 
a=ay 


which, in turn, implies that Gg must be some function of Gmt. 


> Show that the ML estimator ¢ given in (31.71) is an efficient estimator of the parameter t. 


As shown in (31.70), the log-likelihood function in this case is 


N 


In L(x;Tt) =--)- (int+ =). 


i=1 
Differentiating twice with respect to t, we find 


@mnL SX/1 2x \ N ae 
ar? Y (a =) (: pe : Gey) 


i=1 


and so the expectation value of this expression is 


Ink N 2 N 
e| br? ? (1 =E(«l) T° 


where we have used the fact that E[x] = t. Setting b = 0 in (31.18), we thus find that for 
any unbiased estimator of t, 


Vial > 


Zl 


From (31.76), we see that the ML estimator t = 5°, x;/N is unbiased. Moreover, using 
the fact that V[x] = 7°, it follows immediately from (31.40) that V[t] = t?/N. Thus 7 is a 
minimum-variance estimator of t. < 
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31.5.4 Standard errors and confidence limits on ML estimators 


The ML method provides a procedure for obtaining a particular set of estimators 
am for the parameters a of the assumed population P(x|a). As for any other set 
of estimators, the associated standard errors, covariances and confidence intervals 
can be found as described in subsections 31.3.3 and 31.3.4. 


>A company measures the duration (in minutes) of the 10 intervals x;, i = 1,2,...,10, 
between successive telephone calls made to its switchboard to be as follows: 


(ass wh aie LGB} Lil 6S 8S ON) SO D2), 


Supposing that the sample values are drawn independently from the probability distribution 
P(x|t) = (1/t) exp(—x/t), find the ML estimate of the mean t and quote an estimate of 
the standard error on your result. 


As shown in (31.71) the (unbiased) ML estimator 7 in this case is simply the sample mean 
xX = 3.77. Also, as shown in subsection 31.5.3, t is a minimum-variance estimator with 
V[t] =7°/N. Thus, the standard error in 7 is simply 


aa (31.78) 


Since we do not know the true value of t, however, we must instead quote an estimate 6G; 
of the standard error, obtained by substituting our estimate t for t in (31.78). Thus, we 
quote our final result as 

tatt— =377 4119. (31.79) 


JN 


For comparison, the true value used to create the sample was t = 4. < 


For the particular problem considered in the above example, it is in fact possible 
to derive the full sampling distribution of the ML estimator t using characteristic 
functions, and it is given by 


. NN 78-1 Nt 
P(@|t) = (Wat exp ( = ). (31.80) 


where N is the size of the sample. This function is plotted in figure 31.7 for the 
case t = 4 and N = 10, which pertains to the above example. Knowledge of the 
analytic form of the sampling distribution allows one to place confidence limits 
on the estimate 7 obtained, as discussed in subsection 31.3.4. 


> Using the sample values in the above example, obtain the 68% central confidence interval 
on the value of t. 


For the sample values given, our observed value of the ML estimator is 7,,, = 3.77. Thus, 
from (31.28) and (31.29), the 68% central confidence interval [t_,t,] on the value of t is 
found by solving the equations 


tobs 
/ P(t|t,) dt = 0.16, 


00 


: P(4\t_)dt = 0.16, 


Tobs 
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Figure 31.7 The sampling distribution P(@|t) for the estimator t for the case 
t=4and N = 10. 


where P(#|t) is given by (31.80) with N = 10. The above integrals can be evaluated 
analytically but the calculations are rather cumbersome. It is much simpler to evaluate 
them by numerical integration, from which we find [t_,t4] = [2.86,5.46]. Alternatively, 
we could quote the estimate and its 68% confidence interval as 


CS 37S. 


Thus we see that the 68% central confidence interval is not symmetric about the estimated 
value, and differs from the standard error calculated above. This is a result of the (non- 
Gaussian) shape of the sampling distribution P(7|t), apparent in figure 31.7. < 


In many problems, however, it is not possible to derive the full sampling 
distribution of an ML estimator @ in order to obtain its confidence intervals. 
Indeed, one may not even be able to obtain an analytic formula for its standard 
error oq. This is particularly true when one is estimating several parameter 4 
simultaneously, since the joint sampling distribution will be, in general, very 
complicated. Nevertheless, as we discuss below, the likelihood function L(x;a) 
itself can be used very simply to obtain standard errors and confidence intervals. 
The justification for this has its roots in the Bayesian approach to statistics, as 
opposed to the more traditional frequentist approach we have adopted here. We 
now give a brief discussion of the Bayesian viewpoint on parameter estimation. 


31.5.5 The Bayesian interpretation of the likelihood function 


As stated at the beginning of section 31.5, the likelihood function L(x;a) is 
defined by 


P(x\a, H) = L(x;a), 
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where H denotes our hypothesis of an assumed functional form. Now, using 
Bayes’ theorem (see subsection 30.2.3), we may write 


P(x\a, H)P(al|H) 


P(a|x, H) = P(x) > 


(31.81) 
which provides us with an expression for the probability distribution P(a|x, H) 
of the parameters a, given the (fixed) data x and our hypothesis H, in terms of 
other quantities that we may assign. The various terms in (31.81) have special 
formal names, as follows. 


e The quantity P(a|H) on the RHS is the prior probability, which represents our 
state of knowledge of the parameter values (given the hypothesis H) before we 
have analysed the data. 

e This probability is modified by the experimental data x through the likelihood 
P(x\a, H). 

e When appropriately normalised by the evidence P(x|H), this yields the posterior 
probability P(a|x,H), which is the quantity of interest. 

e The posterior encodes all our inferences about the values of the parameters a. 
Strictly speaking, from a Bayesian viewpoint, this entire function, P(a|x, H), is 
the ‘answer’ to a parameter estimation problem. 


Given a particular hypothesis, the (normalising) evidence factor P(x|H) is 
unimportant, since it does not depend explicitly upon the parameter values a. 
Thus, it is often omitted and one considers only the proportionality relation 


P(a|x, H) oc P(x\a, H)P(a|H). (31.82) 


If necessary, the posterior distribution can be normalised empirically, by requiring 
that it integrates to unity, ie. [ P(a|x, H) da = 1, where the integral extends over 
all values of the parameters aj, d2,...,dm. 

The prior P(a|H) in (31.82) should reflect our entire knowledge concerning the 
values of the parameters a, before the analysis of the current data x. For example, 
there may be some physical reason to require some or all of the parameters to 
lie in a given range. If we are largely ignorant of the values of the parameters, 
we often indicate this by choosing a uniform (or very broad) prior, 


P(a|H) = constant, 


in which case the posterior distribution is simply proportional to the likelihood. 
In this case, we thus have 


P(a|x, H) oc L(x;a). (31.83) 


In other words, if we assume a uniform prior then we can identify the posterior 
distribution (up to a normalising factor) with L(x;a), considered as a function of 
the parameters a. 
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Thus, a Bayesian statistician considers the ML estimates Gy, of the parameters 
to be the values that maximise the posterior P(a|x,H) under the assumption of 
a uniform prior. More importantly, however, a Bayesian would not calculate the 
standard error or confidence interval on this estimate using the (classical) method 
employed in subsection 31.3.4. Instead, a far more straightforward approach is 
adopted. Let us assume, for the moment, that one is estimating just a single 
parameter a. Using (31.83), we may determine the values a_ and a; such that 


Pr(a < a_|x, H) = / - L(x;a) da = 4, 


—0oo 


Pr(a > a,|x, H) = / " L(x;a)da =f. 
ay 
where it is assumed that the likelihood has been normalised in such a way that 
f L(x;a) da = 1. Combining these equations gives 


a. 
Pra. <a<aylx,H) = | ava 1-—a—-f, (31.84) 
a_ 
and [a_,a,] is the Bayesian confidence interval on the value of a at the confidence 
level 1 — «— f. As in the case of classical confidence intervals, one often quotes 
the central confidence interval, for which « = 8. Another common choice (where 
possible) is to use the two values a_ and a, satisfying (31.84), for which L(x;a_) = 
L(x; a4). 

It should be understood that a frequentist would consider the Bayesian confi- 
dence interval as an approximation to the (classical) confidence interval discussed 
in subsection 31.3.4. Conversely, a Bayesian would consider the confidence inter- 
val defined in (31.84) to be the more meaningful. In fact, the difference between 
the Bayesian and classical confidence intervals is rather subtle. The classical con- 
fidence interval is defined in such a way that if one took a large number of 
samples each of size N and constructed the confidence interval in each case then 
the proportion of cases in which the true value of a would be contained within the 
interval is 1—a«—f. For the Bayesian confidence interval, one does not rely on the 
frequentist concept of a large number of repeated samples. Instead, its meaning is 
that, given the single sample x (and our hypothesis H for the functional form of 
the population), the probability that a lies within the interval [a_,a;] is 1—a«—f. 

By adopting the Bayesian viewpoint, the likelihood function L(x;a) may also 
be used to obtain an approximation 6, to the standard error in the ML estimator; 
the approximation is given by 


@inL| \ 7? 
6, = (| - —— : 31.85 

~ ( 0a’ |) ( 
Clearly, if L(x;a) were a Gaussian centred on a = @ then Gq would be its standard 


deviation. Indeed, in this case, the resulting ‘one-sigma’ limits would constitute a 
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Figure 31.8 The likelihood function L(x;t) (normalised to unit area) for the 
sample values given in the worked example in subsection 31.5.4 and indicated 
here by short vertical lines. 


68.3% Bayesian central confidence interval. Even when L(x; a) is not Gaussian, 
however, (31.85) is often used as a measure of the standard error. 


> For the sample data given in subsection 31.5.4, use the likelihood function to estimate the 
standard error 6; in the ML estimator % and obtain the Bayesian 68% central confidence 
interval on t. 


We showed in (31.67) that the likelihood function in this case is given by 


1 1 
L(x;t) = aw eXP |-z +x2+-+++ x9] ; 


where x;, i= 1,2,...,N, denotes the sample value and N = 10. This likelihood function is 
plotted in figure 31.8, after normalising (numerically) to unit area. The short vertical lines 
in the figure indicate the sample values. We see that the likelihood function peaks at the 
ML estimate t = 3.77 that we found in subsection 31.5.4. Also, from (31.77), we have 


Ink N? 2< 
1 x; } . 
Ot? T tN > s 
Remembering that t = >, x;/N, our estimate of the standard error in 7 is 
inL| \"? ¢ 
a= (-S ) Sy 48. 


Or? 
which is precisely the estimate of the standard error we obtained in subsection 31.5.4. 
It should be noted, however, that in general we would not expect the two estimates of 
standard error made by the different methods to be identical. 

In order to calculate the Bayesian 68% central confidence interval, we must determine 
the values a_ and a, that satisfy (31.84) with « = f = 0.16. In this case, the calculation 
can be performed analytically but is somewhat tedious. It is trivial, however, to determine 
a_ and a; numerically and we find the confidence interval to be [3.16,6.20]. Thus we can 
quote our result with 68% central confidence limits as 


$= 371 
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By comparing this result with that given towards the end of subsection 31.5.4, we see that, 
as we might expect, the Bayesian and classical confidence intervals differ somewhat. < 


The above discussion is generalised straightforwardly to the estimation of 


several parameters a1, d7,...,day, simultaneously. The elements of the inverse of 
the covariance matrix of the ML estimators can be approximated by 
A2: 
0“ InL 
(V"')y;=-3=— | (31.86) 
0aj0aj; ak 


From (31.36), we see that (at least for unbiased estimators) the expectation value 
of (31.86) is equal to the element Fj; of the Fisher matrix. 

The construction of a multi-dimensional Bayesian confidence region is also 
straightforward. For a given confidence level 1 — « (say), it is most common 
to construct the confidence region as the M-dimensional region R in a-space, 
bounded by the ‘surface’ L(x;a) = constant, for which 


if L(x;a)d“a =1-4, 
R 


where it is assumed that L(x;a) is normalised to unit volume. Moreover, we 
see from (31.83) that (assuming a uniform prior probability) we may obtain the 
marginal posterior distribution for any parameter a; simply by integrating the 
likelihood function L(x;a) over the other parameters: 


Paix) = fof Loxsa) day -+-da.dayes ++ day 


Here the integral extends over all possible values of the parameters, and again 
is it assumed that the likelihood function is normalised in such a way that 
f L(x;a)d“a = 1. This marginal distribution can then be used as above to 
determine Bayesian confidence intervals on each a; separately. 


» Ten independent sample values x;, i= 1,2,...,10, are drawn at random from a Gaussian 
distribution with unknown mean p and standard deviation o. The sample values are as 
follows (to two decimal places): 


2.22 2.56 107 0.24 018 0.95 0.73 —0.79 2.09 1.81 


Find the Bayesian 95% central confidence intervals on w and o separately. 


The likelihood function in this case is 
1 
L(x3p,0) = (2107)? exp -s (x; — " ; (31.87) 
o 


Assuming uniform priors on w and o (over their natural ranges of -o0 — o and0 > oc 
respectively), we may identify this likelihood function with the posterior probability, as in 
(31.83). Thus, the marginal posterior distribution on y is given by 


same ne 5 
PuilxHt) x [ ow exP 2s Yoo do. 
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By substituting ¢ = 1/u (so that do = —du/u’) and integrating by parts either (N — 2)/2 
or (N — 3)/2 times, we find 


P(ulx,H) oc [N(&— pw)? +NS] OO, 


where we have used the fact that 3>,(x;— 4)? = N(X— ») + Ns’, X being the sample mean 
and s? the sample variance. We may now obtain the 95% central confidence interval by 
finding the values w_ and yw, for which 


Pes (oe) 
/ P(u|x, H) du = 0.025 and / P(u|x, H) du = 0.025. 


00 fn 


The normalisation of the posterior distribution and the values w— and wy, are easily 
obtained by numerical integration. Substituting in the appropriate values N = 10, x = 1.11 
and s = 1.01, we find the required confidence interval to be [0.29, 1.97]. 

To obtain a confidence interval on o, we must first obtain the corresponding marginal 
posterior distribution. From (31.87), again using the fact that 37 ,(x;—p)? = N(x—p)°+Ns°, 
this is given by 


1 Ns? © N(x— py? 
P(a|x, H) x Gow XP (-3) 4 exp a du. 


Noting that the integral of a one-dimensional Gaussian is proportional to o, we conclude 
that 


1 Ns? 
P(a|x, H) «< —- exp (-3) : 
o 


The 95% central confidence interval on o can then be found in an analogous manner to 
that on yw, by solving numerically the equations 


i i P(o|x,H)do = 0.025 and /  P(olx,H) do = 0.025. 
0 


ot 


We find the required interval to be [0.76, 2.16]. < 


31.5.6 Behaviour of ML estimators for large N 


As mentioned in subsection 31.3.6, in the large-sample limit N — oo, the sampling 
distribution of a set of (consistent) estimators a, whether ML or not, will tend, 
in general, to a multivariate Gaussian centred on the true values a. This is a 
direct consequence of the central limit theorem. Similarly, in the limit N — oo the 
likelihood function L(x;a) also tends towards a multivariate Gaussian but one 
centred on the ML estimate(s) a. Thus ML estimators are always asymptotically 
consistent. This limiting process was illustrated for the one-dimensional case by 
figure 31.5. 
Thus, as N becomes large, the likelihood function tends to the form 


L(x; a) = Linax exp [—$0(a, a)| > 
where Q denotes the quadratic form 
O(a, 4) = (a—a)'V—!(a — a) 
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and the matrix V~! is given by 


(Ve = 


Moreover, in the limit of large N, this matrix tends to the Fisher matrix given in 
(31.36), ie. V-! > F. Hence ML estimators are asymptotically minimum-variance. 

Comparison of the above results with those in subsection 31.3.6 shows that 
the large-sample limit of the likelihood function L(x;a) has the same form as the 
large-sample limit of the joint estimator sampling distribution P(aja). The only 
difference is that P(ala) is centred in a-space on the true values 4 = a whereas 
L(x;a) is centred in a-space on the ML estimates a = 4. From figure 31.4 and its 
accompanying discussion, we therefore conclude that, in the large-sample limit, 
the Bayesian and classical confidence limits on the parameters coincide. 


67 In L 


Oaj0a; a=a : 


31.5.7 Extended maximum-likelihood method 


It is sometimes the case that the number of data items N in our sample is itself a 
random variable. Such experiments are typically those in which data are collected 
for a certain period of time during which events occur at random in some way, 
as opposed to those in which a prearranged number of data items are collected. 
In particular, let us consider the case where the sample values x1, X2,...,xn are 
drawn independently from some distribution P(x|a) and the sample size N is a 
random variable described by a Poisson distribution with mean /, ie. N ~ Po(A). 
The likelihood function in this case is given by 
HO ag 
L(x, Ns 4,a) = se [J Poa), (31.88) 
i=1 


and is often called the extended likelihood function. The function L(x;/,a) can 
be used as before to estimate parameter values or obtain confidence intervals. 
Two distinct cases arise in the use of the extended likelihood function, depending 
on whether the Poisson parameter / is a function of the parameters a or is an 
independent parameter. 
Let us first consider the case in which J is a function of the parameters a. From 

(31.88), we can write the extended log-likelihood function as 

N N 

In L = N In A(a) — A(a) + > In P(xila) = —A(a) + S> In[A(@)P (xila)]- 

ol i=] 
where we have ignored terms not depending on a. The ML estimates 4 of the 
parameters can then be found in the usual way, and the ML estimate of the 
Poisson parameter is simply 4. = ia). The errors on our estimators 4 will be, in 
general, smaller than those obtained in the usual likelihood approach, since our 
estimate includes information from the value of N as well as the sample values x;. 
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The other possibility is that 2 is an independent parameter and not a function 
of the parameters a. In this case, the extended log-likelihood function is 


N 
InL =NiInd—A+5— In P(xjla), (31.89) 
i=1 


4 


where we have omitted terms not depending on 4 or a. Differentiating with 
respect to 4 and setting the result equal to zero, we find that the ML estimate of 
4 is simply 

A=N. 


By differentiating (31.89) with respect to the parameters a; and setting the results 
equal to zero, we obtain the usual ML estimates 4; of their values. In this case, 
however, the errors in our estimates will be larger, in general, than those in the 
standard likelihood approach, since they must include the effect of statistical 
uncertainty in the parameter /. 


31.6 The method of least squares 


The method of least squares is, in fact, just a special case of the method of 
maximum likelihood. Nevertheless, it is so widely used as a method of parameter 
estimation that it has acquired a special name of its own. At the outset, let us 
suppose that a data sample consists of a set of pairs (x;,y;), i = 1,2,...,N. For 
example, these data might correspond to the temperature y; measured at various 
points x; along some metal rod. 

For the moment, we will suppose that the x; are known exactly, whereas there 
exists a measurement error (or noise) n; on each of the values y;. Moreover, let 
us assume that the true value of y at any position x is given by some function 
y =f(x;a) that depends on the M unknown parameters a. Then 


yi = f (xia) + Nj. 


Our aim is to estimate the values of the parameters a from the data sample. 

Bearing in mind the central limit theorem, let us suppose that the n; are drawn 
from a Gaussian distribution with no systematic bias and hence zero mean. In the 
most general case the measurement errors n; might not be independent but be 
described by an N-dimensional multivariate Gaussian with non-trivial covariance 
matrix N, whose elements Njj = Cov[n;,nj] we assume to be known. Under these 
assumptions it follows from (30.148), that the likelihood function is 


L(x, y3a) = exp [—37°(a)], 


1 
(27)N/2|NI1/2 
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where the quantity denoted by 7’ is given by the quadratic form 
N 
La) = So bi-—fxisalN-Yijlyj — f0xjsa)] = (y— 8) Ny — f. 

j=l (31.90) 
In the last equality, we have rewritten the expression in matrix notation by 
defining the column vector f with elements f; = f(x;;a). We note that in the 
(common) special case in which the measurement errors n; are independent, their 
covariance matrix takes the diagonal form N = diag(o7,03,...,0%), where oj is 
the standard deviation of the measurement error n;. In this case, the expression 
(31.90) for y* reduces to 


N 
Pies: pee = fs ey, 
i=1 

The least-squares (LS) estimators 4,5 of the parameter values are defined as 
those that minimise the value of 77(a); they are usually determined by solving 

the M equations 
077 
Oa; 


=0 fori=1,2....,M. (31.91) 


a=A_s 

Clearly, if the measurement errors n; are indeed Gaussian distributed, as assumed 
above, then the LS and ML estimators of the parameters a coincide. Because 
of its relative simplicity, the method of least squares is often applied to cases in 
which the n; are not Gaussian distributed. The resulting estimators ays are not the 
ML estimators, and the best that can be said in justification is that the method is 
an obviously sensible procedure for parameter estimation that has stood the test 
of time. 

Finally, we note that the method of least squares is easily extended to the case 
in which each measurement y; depends on several variables, which we denote 
by x;. For example, y; might represent the temperature measured at the (three- 
dimensional) position x; in a room. In this case, the data is modelled by a 
function y = f(x;;a), and the remainder of the above discussion carries through 
unchanged. 


31.6.1 Linear least squares 


We have so far made no restriction on the form of the function f(x;a). It so 
happens, however, that, for a model in which f(x;a) is a linear function of the 
parameters a),d,...,dy, one can always obtain analytic expressions for the LS 
estimators 4ys and their variances. The general form of this kind of model is 

M 


f (xia) = S° aihi(x), (31.92) 


i=1 
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where {h1(x), h2(x),...,4a(x)} is some set of linearly independent fixed functions 
of x, often called the basis functions. Note that the functions h;(x) themselves may 
be highly non-linear functions of x. The ‘linear’ nature of the model (31.92) refers 
only to its dependence on the parameters a;. Furthermore, in this case, it may 
be shown that the LS estimators G; have zero bias and are minimum-variance, 
irrespective of the probability density function from which the measurement errors 
n,; are drawn. 

In order to obtain analytic expressions for the LS estimators ays, it is convenient 
to write (31.92) in the form 


M 
f(xsa) = 5 Riyaj, (31.93) 
j=l 


where R;; = h;(x;) is an element of the response matrix R of the experiment. The 
expression for 7? given in (31.90) can then be written, in matrix notation, as 


x (a) = (y — Ra)"N7!(y — Ra). (31.94) 


The LS estimates of the parameters a are now found, as shown in (31.91), by 
differentiating (31.94) with respect to the a; and setting the resulting expressions 
equal to zero. Denoting by V7? the vector with elements 0y7/éa;, we find 


V7 = —2R™N7|(y — Ra). (31.95) 


This can be verified by writing out the expression (31.94) in component form and 
differentiating directly. 


> Verify result (31.95) by formulating the calculation in component form. 


To make the derivation less cumbersome, let us adopt the summation convention discussed 
in section 26.1, in which it is understood that any subscript that appears exactly twice in 
any term of an expression is to be summed over all the values that a subscript in that 
position can take. Thus, writing (31.94) in component form, we have 


La) = (vi — Rag )(N~)ij(¥j — Raa). 


Differentiating with respect to a, gives 


2: 


Ox 


Ba Ri dip(N ivy — Ryan) + (Vi — Rica (N~)i(—Rjdip) 
edd 


= —R,(N7')ij(yj — Ryan) — (i — Rite) (N~')ij Rip, (31.96) 


where 6,; is the Kronecker delta symbol discussed in section 26.1. By swapping the indices 
i and j in the second term on the RHS of (31.96) and using the fact that the matrix N7! 
is symmetric, we obtain 
Ay2 
ox -1 
—— =—-2Ri(N7 )ij(yj — Ria 
Oa, p( id; ik Ak) 
= —2(R")pi(N~")ij(¥j — Rika). (31.97) 


If we denote the vector with components 077/da,, p = 1,2,...,M, by Vz’ and write the 
RHS of (31.97) in matrix notation, we recover the result (31.95). < 
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Setting the expression (31.95) equal to zero at a = 4, we find 
—2R™N7!y + 2R™N7!RA = 0. 


Provided the matrix R'N7!R is not singular, we may solve this equation for 4 to 
obtain 


4 =(R'N7'R)'!R'N7!y = Sy, (31.98) 


thus defining the Mx N matrix S. It follows that the LS estimates @;, i = 1,2,...,M, 
are linear functions of the original measurements y;, j = 1,2,...,N. Moreover, 
using the error propagation formula (30.141) derived in subsection 30.12.3, we 
find that the covariance matrix of the estimators @; is given by 


V = Cov[a;,4)] = SNS’ = (R™N“!R)!. (31.99) 


The two equations (31.98) and (31.99) contain the complete method of least 
squares. In particular, we note that, if one calculates the LS estimates using 
(31.98) then one has already obtained their covariance matrix (31.99). 


[> Prove result (31.99). 


Using the definition of S given in (31.98), the covariance matrix (31.99) becomes 
V = SNST 
= [(R™N“'R)RTN7!JN[(RTN7'R) RINT! YT. 
Using the result (AB---C)' =C!.--B™A! for the transpose of a product of matrices and 
noting that, for any non-singular matrix, (A~!)™ = (A‘)~! we find 
V = (R™NR)-'R™N!N(N')R[(RIN RY] 
= (R™N7'R)'R™N7!R(R™N7!R)! 
= (RNR) 


where we have also used the fact that N is symmetric and so N' =N. 4 


It is worth noting that one may also write the elements of the (inverse) 


covariance matrix as 
1/ &/ 
1 § 
(Vig = 5 


2 ee 
which is the same as the Fisher matrix (31.36) in cases where the measurement 
errors are Gaussian distributed (and so the log-likelihood is In L = —y7/2). This 
proves, at least for this case, our earlier statement that the LS estimators are 
minimum-variance. In fact, since f(x;a) is linear in the parameters a, one can 
write x? exactly as 


; ‘ 1 02? 
2a) = 20) +590 (GE) (a Anta —a), 


aj0a; 


ij=l 


which is quadratic in the parameters a;. Hence the form of the likelihood function 
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Figure 31.9 A set of data points with error bars indicating the uncertainty 
o = 0.5 on the y-values. The straight line is y = mx + ¢, where mm and ¢ are 
the least-squares estimates of the slope and intercept. 


L « exp(—7?/2) is Gaussian. From the discussions of subsections 31.3.6 and 
31.5.6, it follows that the ‘surfaces’ y7(a) = c, where c is a constant, bound 
ellipsoidal confidence regions for the parameters a;. The relationship between the 
value of the constant c and the confidence level is given by (31.39). 


> An experiment produces the following data sample pairs (x;, y;): 


1.85 2.72 281 3.06 342 3.76 431 447 464 4.99 
2.26 3.10 3.80 411 474 431 5.24 403 5.69 6.57 


where the x;-values are known exactly but each y;-value is measured only to an accuracy 


of o = 0.5. Assuming the underlying model for the data to be a straight line y = mx +c, 
find the LS estimates of the slope m and intercept c and quote the standard error on each 
estimate. 


The data are plotted in figure 31.9, together with error bars indicating the uncertainty in 
the y;-values. Our model of the data is a straight line, and so we have 
f(xse,m) =c+mx. 


In the language of (31.92), our basis functions are h,(x) = 1 and h2(x) = x and our model 
parameters are a; = c and a) = m. From (31.93) the elements of the response matrix are 
Ri = hj(x;), so that 


1 x 
1 x 

R=]. . J], (31.100) 
1) seg 


where x; are the data values and N = 10 in our case. Further, since the standard deviation 
on each measurement error is ¢, we have N = o7l, where | is the N x N identity matrix. 
Because of this simple form for N, the expression (31.98) for the LS estimates reduces to 
as 1 
@ = 0°(R™R) '—R'y =(R'R) 'R’y. (31.101) 
o 


Note that we cannot expand the inverse in the last line, since R itself is not square and 
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hence does not possess an inverse. Inserting the form for R in (31.100) into the expression 


(31.101), we find 
€ )=( Dil Lixi ) (ee ) 
he} \ Six ax? Vix 


_ 1 x x Ny 
~ NG2—R)\ -X¥ 1 Nxy }° 


We thus obtain the LS estimates 


SUEY gad Ge ee ee (31.102) 


n= > 
x2 — x2 x2 — x2 


where the last expression for ¢ shows that the best-fit line passes through the ‘centre 
of mass’ (x, jy) of the data sample. To find the standard errors on our results, we must 
calculate the covariance matrix of the estimators. This is given by (31.99), which in our 
case reduces to 


2 Za = 
Vor Ria == ee), (31.103) 
N(x2 — x2) \ -* 1 


The standard error on each estimator is simply the positive square root of the corresponding 
diagonal element, i.e. og = \/Vi; and oj, = \/Vx, and the covariance of the estimators m 
and ¢ is given by Cov[¢,m] = Vi. = V2. Inserting the data sample averages and moments 
into (31.102) and (31.103), we find 


c=C+a, = 0.40 + 0.62 and m=mtom = 111+40.17. 


The ‘best-fit’ straight line y = mx + ¢ is plotted in figure 31.9. For comparison, the true 
values used to create the data were m= 1 andc=1. <4 


The extension of the method to fitting data to a higher-order polynomial, such 
as f(x;a) =a, +aox+ a3x*, is obvious. However, as the order of the polynomial 
increases the matrix inversions become rather complicated. Indeed, even when the 
matrices are inverted numerically, the inversion is prone to numerical instabilities. 
A better approach is to replace the basis functions h,,(x) = x”, m = 1,2,...,M, 
with a set of polynomials that are ‘orthogonal over the data’, i.e. such that 


N 
So M(xin(x)) =0 for 14m. 
i=1 


Such a set of polynomial basis functions can always be found by using the Gram— 
Schmidt orthogonalisation procedure presented in section 17.1. The details of this 
approach are beyond the scope of our discussion but we note that, in this case, 
the matrix R™R is diagonal and may be inverted easily. 


31.6.2 Non-linear least squares 


If the function f(x;a) is not linear in the parameters a then, in general, it is 
not possible to obtain an explicit expression for the LS estimates a. Instead, one 
must use an iterative (numerical) procedure, which we now outline. In practice, 
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however, such problems are best solved using one of the many commercially 
available software packages. 

One begins by making a first guess a° for the values of the parameters. At this 
point in parameter space, the components of the gradient Vz’ will not be equal 
to zero, in general (unless one makes a very lucky guess!). Thus, for at least some 
values of i, we have 


7 y2 
Sl Wg 5 


04a; a=al’ 


Our aim is to find a small increment da in the values of the parameters, such that 


07 


esis! =0 for all i. (31.104) 
0a; 


a=a+da 


If our first guess a° were sufficiently close to the true (local) minimum of 7’, 


we could find the required increment 6a by expanding the LHS of (31.104) as a 
Taylor series about a = a°, keeping only the zeroth-order and first-order terms: 


M 
077 


x 
- D Oaj0a; 
j=l 


Setting this expression to zero, we find that the increments da; may be found by 
solving the set of M linear equations 


07 


oy’ 
04a; 


04a; 


5aj. (31.105) 


a=a? 


a=a+da a=a? 


3 022 ; 0/2 
aj=— : 
jal 6a;0a; a=a? J 0a; eae 


It most cases, however, our first guess a° will not be sufficiently close to the true 
minimum for (31.105) to be an accurate approximation, and consequently (31.104) 
will not be satisfied. In this case, a! = a® + da is (hopefully) an improved guess 
at the parameter values; the whole process is then repeated until convergence is 
achieved. 

It is worth noting that, when one is estimating several parameters a, the 
function 7?(a) may be very complicated. In particular, it may possess numerous 
local extrema. The procedure outlined above will converge to the local extremum 
‘nearest’ to the first guess a?. Since, in fact, we are interested only in the local 
minimum that has the absolute lowest value of y7(a), it is clear that a large part 
of solving the problem is to make a ‘good’ first guess. 


31.7 Hypothesis testing 


So far we have concentrated on using a data sample to obtain a number or a set 
of numbers. These numbers may be estimated values for the moments or central 
moments of the population from which the sample was drawn or, more generally, 
the values of some parameters a in an assumed model for the data. Sometimes, 
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however, one wishes to use the data to give a ‘yes’ or ‘no’ answer to a particular 
question. For example, one might wish to know whether some assumed model 
does, in fact, provide a good fit to the data, or whether two parameters have the 
same value. 


31.7.1 Simple and composite hypotheses 


In order to use data to answer questions of this sort, the question must be 
posed precisely. This is done by first asserting that some hypothesis is true. 
The hypothesis under consideration is traditionally called the null hypothesis 
and is denoted by Hp. In particular, this usually specifies some form P(x|Ho) 
for the probability density function from which the data x are drawn. If the 
hypothesis determines the PDF uniquely, then it is said to be a simple hypothesis. 
If, however, the hypothesis determines the functional form of the PDF but not the 
values of certain parameters a on which it depends then it is called a composite 
hypothesis. 

One decides whether to accept or reject the null hypothesis Hp by performing 
some statistical test, as described below in subsection 31.7.2. In fact, formally 
one uses a Statistical test to decide between the null hypothesis Hp and the 
alternative hypothesis H,. We define the latter to be the complement Hy of the 
null hypothesis within some restricted hypothesis space known (or assumed) in 
advance. Hence, rejection of Hp implies acceptance of H;, and vice versa. 

As an example, let us consider the case in which a sample x is drawn from a 
Gaussian distribution with a known variance o? but with an unknown mean u. 
If one adopts the null hypothesis Hp that u = 0, which we write as Ho : u = 0, 
then the corresponding alternative hypothesis must be H, : u # 0. Note that, 
in this case, Hp is a simple hypothesis whereas H, is a composite hypothesis. 
If, however, one adopted the null hypothesis Hp : « < 0 then the alternative 
hypothesis would be H; : u > 0, so that both Ho and H; would be composite 
hypotheses. Very occasionally both Hp and H, will be simple hypotheses. In our 
illustration, this would occur, for example, if one knew in advance that the mean 
pe of the Gaussian distribution were equal to either zero or unity. In this case, if 
one adopted the null hypothesis Ho : u = 0 then the alternative hypothesis would 
be Hy :u=1. 


31.7.2 Statistical tests 


In our discussion of hypothesis testing we will restrict our attention to cases in 
which the null hypothesis Ho is simple (see above). We begin by constructing a 
test statistic t(x) from the data sample. Although, in general, the test statistic need 
not be just a (scalar) number, and could be a multi-dimensional (vector) quantity, 
we will restrict our attention to the former case. Like any statistic, t(x) will be a 


1278 


31.7 HYPOTHESIS TESTING 


P(t\|Ho) 
a 
t 
Corit 
P(t|H;) 
e ~ 
t 
lorit 


Figure 31.10 The sampling distributions P(t|Ho) and P(t|H,) of a test statistic 
t. The shaded areas indicate the (one-tailed) regions for which Pr(t > terit|Ho) = 
a and Pr(t < teit|H1) = BP respectively. 


random variable. Moreover, given the simple null hypothesis Hp concerning the 
PDF from which the sample was drawn, we may determine (in principle) the 
sampling distribution P(t|Ho) of the test statistic. A typical example of such a 
sampling distribution is shown in figure 31.10. One defines for t a rejection region 
containing some fraction « of the total probability. For example, the (one-tailed) 
rejection region could consist of values of t greater than some value ty it, for 
which 
Pr(t > terit|Ho) = : P(t|Ho) dt = «; (31.106) 
Crit 
this is indicated by the shaded region in the upper half of figure 31.10. Equally, 
a (one-tailed) rejection region could consist of values of t less than some value 
terit- Alternatively, one could define a (two-tailed) rejection region by two values 
ty and ft) such that Pr(t; < t < t)|Ho) =a. In all cases, if the observed value of t 
lies in the rejection region then Hp is rejected at significance level «; otherwise Ho 
is accepted at this same level. 
It is clear that there is a probability « of rejecting the null hypothesis Ho 
even if it is true. This is called an error of the first kind. Conversely, an error 
of the second kind occurs when the hypothesis Hp is accepted even though it is 
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false (in which case H, is true). The probability 6 (say) that such an error will 
occur is, in general, difficult to calculate, since the alternative hypothesis H is 
often composite. Nevertheless, in the case where H; is a simple hypothesis, it is 
straightforward (in principle) to calculate 6. Denoting the corresponding sampling 
distribution of t by P(t|H,), the probability 6 is the integral of P(t|H,) over the 
complement of the rejection region, called the acceptance region. For example, in 
the case corresponding to (31.106) this probability is given by 


Corit 


B = Prt < tectlH) = i} P(t\Hy) dt. 


This is illustrated in figure 31.10. The quantity 1 — f is called the power of the 
statistical test to reject the wrong hypothesis. 


31.7.3 The Neyman-—Pearson test 


In the case where Hp and H; are both simple hypotheses, the Neyman—Pearson 
lemma (which we shall not prove) allows one to determine the ‘best’ rejection 
region and test statistic to use. 

We consider first the choice of rejection region. Even in the general case, in 
which the test statistic t is a multi-dimensional (vector) quantity, the Neyman— 
Pearson lemma states that, for a given significance level «, the rejection region for 
Ho giving the highest power for the test is the region of t-space for which 


P(t|Ho) 


Pal)? (31.107) 


where c is some constant determined by the required significance level. 

In the case where the test statistic t is a simple scalar quantity, the Neyman— 
Pearson lemma is also useful in deciding which such statistic is the ‘best’ in 
the sense of having the maximum power for a given significance level «. From 
(31.107), we can see that the best statistic is given by the likelihood ratio 


mace) (31.108) 
P(x|H1) 
and that the corresponding rejection region for Ho is given by t < fei. In fact, 
it is clear that any statistic u = f(t) will be equally good, provided that f(t) is a 
monotonically increasing function of t. The rejection region is then u < f (tert). 
Alternatively, one may use any test statistic v = g(t) where g(t) is a monotonically 
decreasing function of t; in this case the rejection region becomes v > g(terit). To 
construct such statistics, however, one must know P(x|Ho) and P(x|H,) explicitly, 


and such cases are rare. 
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> Ten independent sample values x;, i= 1,2,...,10, are drawn at random from a Gaussian 
distribution with standard deviation 6 = 1. The mean wu of the distribution is known to 
equal either zero or unity. The sample values are as follows: 


2.22 2.56 1.07 0.24 018 0.95 073 —0.79 2.09 1.81 


Test the null hypothesis Ho : 1 = 0 at the 10% significance level. 


The restricted nature of the hypothesis space means that our null and alternative hypotheses 
are Hy :=0 and H; : uy = 1 respectively. Since Hp and H; are both simple hypotheses, 
the best test statistic is given by the likelihood ratio (31.108). Thus, denoting the means 
by fo and 441, we have 


exp [=3 Ubi = Ho)] _ exp [=3 Vike = 2Hoxi + 149)] 
exp [—} DiGi — P| exp[—F LG 2a +19) 
exp [(tlo — #1), 7 — EN(8 — 1)] - 

Inserting the values jo = 0 and py = 1, yields t = exp(—Nx + 3N), where X is the 


sample mean. Since —In t is a monotonically decreasing function of t, however, we may 
equivalently use as our test statistic 


1 in oe 
v=—intts =x, 


where we have divided by the sample size N and added ; for convenience. Thus we 
may take the sample mean as our test statistic. From (31.13), we know that the sampling 
distribution of the sample mean under our null hypothesis Hp is the Gaussian distribution 
N(uo,07/N), where fio = 0, ¢? = 1 and N = 10. Thus x ~ N(0,0.1). 

Since X is a monotonically decreasing function of t, our best rejection region for a given 
significance « is X > Xcrit, Where Xcrir depends on «. Thus, in our case, Xcrit is given by 


a=1-0 (3) = 1— 010%), 


where ®(z) is the cumulative distribution function for the standard Gaussian. For a 10% 
significance level we have « = 0.1 and, from table 30.3 in subsection 30.9.1, we find 
Xerit = 0.128. Thus the rejection region on xX is 


xX > 0.128. 


From the sample, we deduce that x = 1.11, and so we can clearly reject the null hypothesis 
Ho : « = 0 at the 10% significance level. It can, in fact, be rejected at a much higher 
significance level. As revealed on p. 1239, the data was generated using wy = 1. < 


31.7.4 The generalised likelihood-ratio test 


If the null hypothesis Ho or the alternative hypothesis H, is composite (or both 
are composite) then the corresponding distributions P(x|Ho) and P(x|H,) are 
not uniquely determined, in general, and so we cannot use the Neyman—Pearson 
lemma to obtain the ‘best’ test statistic t. Nevertheless, in many cases, there still 
exists a general procedure for constructing a test statistic t which has useful 
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properties and which reduces to the Neyman-Pearson statistic (31.108) in the 
special case where Ho and H; are both simple hypotheses. 

Consider the quite general, and commonly occurring, case in which the 
data sample x is drawn from a population P(x|a) with a known (or as- 
sumed) functional form but depends on the unknown values of some parameters 
a1,d2,...,a4m. Moreover, suppose we wish to test the null hypothesis Ho that 
the parameter values a lie in some subspace S of the full parameter space 
A. In other words, on the basis of the sample x it is desired to test the 
null hypothesis Ho : (a1,a2,...,a lies in S) against the alternative hypothesis 
Hy : (aj, a2,...,ay lies in S), where S is A—S. 

Since the functional form of the population is known, we may write down the 
likelihood function L(x;a) for the sample. Ordinarily, the likelihood will have 
a maximum as the parameters a are varied over the entire parameter space A. 
This is the usual maximum-likelihood estimate of the parameter values, which 
we denote by a. If, however, the parameter values are allowed to vary only over 
the subspace S then the likelihood function will be maximised at the point as, 
which may or may not coincide with the global maximum 4. Now, let us take as 
our test statistic the generalised likelihood ratio 


(31.109) 


where L(x;4s) is the maximum value of the likelihood function in the subspace 
S and L(x;4) is its maximum value in the entire parameter space A. It is clear 
that t is a function of the sample values only and must lie between 0 and 1. 

We will concentrate on the special case where Ho is the simple hypothesis 
Ho : a = ao. The subspace S then consists of only the single point ao. Thus 
(31.109) becomes 


(31.110) 


and the sampling distribution P(t|Ho) can be determined (in principle). As in the 
previous subsection, the best rejection region for a given significance « is simply 
t < teit, Where the value tai, depends on «. Moreover, as before, an equivalent 
procedure is to use as a test statistic u = f(t), where f(t) is any monotonically 
increasing function of t; the corresponding rejection region is then u < f (tert). 
Similarly, one may use a test statistic v = g(t), where g(t) is any monotonically 
decreasing function of t; the rejection region then becomes v > g(fteit). Finally, 
we note that if H, is also a simple hypothesis H; : a = a;, then (31.110) reduces 
to the Neyman-Pearson test statistic (31.108). 
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> Ten independent sample values x;, i= 1,2,...,10, are drawn at random from a Gaussian 
distribution with standard deviation ¢ = 1. The sample values are as follows: 


2.22 2.56 1.07 0.24 018 0.95 073 —0.79 2.09 1.81 


Test the null hypothesis Ho : 1 = 0 at the 10% significance level. 


We must test the (simple) null hypothesis Ho : « = 0 against the (composite) alternative 
hypothesis H, : » #0. Thus, the subspace S is the single point 4 = 0, whereas A is the 
entire p-axis. The likelihood function is 


1 
L(x; ) = Qmyr exp [—4>0,01— 0], 
which has its global maximum at = x. The test statistic t is then given by 
L(x;0 exp (—4)>, x? 
m0) = Pl ey il = exp (—4.Nx’). 
L(x;X) exp [—$0 01 — X)7] 
It is in fact more convenient to consider the test statistic 


v =—2Int = NX’. 


(x) = 


Since —2 Int is a monotonically decreasing function of t, the rejection region now becomes 
UV > Verit, Where 


/ P(v|Ho) dv = a, (31.111) 
Perit 

a being the significance level of the test. Thus it only remains to determine the sampling 
distribution P(v|Ho). Under the null hypothesis Hp, we expect X to be Gaussian distributed, 
with mean zero and variance 1/N. Thus, from subsection 30.9.4, v will follow a chi-squared 
distribution of order 1. Substituting the appropriate form for P(v| Ho) in (31.111) and setting 
a = 0.1, we find by numerical integration (or from table 31.2) that verit = NX it = 2-71, 
Since N = 10, the rejection region on x at the 10% significance level is thus 


xX < —0.52 and & > 0.52. 


As noted before, for this sample x = 1.11, and so we may reject the null hypothesis 
Ho :u~=0 at the 10% significance level. < 


The above example illustrates the general situation that if the maximum- 
likelihood estimates 4 of the parameters fall in or near the subspace S then the 
sample will be considered consistent with Ho and the value of t will be near 
unity. If 4 is distant from S then the sample will not be in accord with Ho and 
ordinarily t will have a small (positive) value. 

It is clear that in order to prescribe the rejection region for t, or for a related 
statistic u or v, it is necessary to know the sampling distribution P(t|Ho). If Ho 
is simple then one can in principle determine P(t|Ho), although this may prove 
difficult in practice. Moreover, if Ho is composite, then it may not be possible 
to obtain P(t|Ho), even in principle. Nevertheless, a useful approximate form for 
P(t|Ho) exists in the large-sample limit. Consider the null hypothesis 


Ho : (ay =a), ay =a),...,aR = a}), where R < M 


and the a? are fixed numbers. (In fact, we may fix the values of any subset 
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containing R of the M parameters.) If Ho is true then it follows from our 
discussion in subsection 31.5.6 (although we shall not prove it) that, when the 
sample size N is large, the quantity —2Int follows approximately a chi-squared 
distribution of order R. 


31.7.5 Student’s t-test 


Student’s t-test is just a special case of the generalised likelihood ratio test applied 
to a sample x1, X2,...,xn drawn independently from a Gaussian distribution for 
which both the mean p and variance o* are unknown, and for which one wishes 
to distinguish between the hypotheses 


Ho : = bo, 0<0° <a, and A, : nF Wo, 0<0 <a, 


where po is a given number. Here, the parameter space A is the half-plane 
—«0 < pw < 0, 0 < o* < ~, whereas the subspace S characterised by the null 
hypothesis Hp is the line pp = 9, 0 < a? < ©. 

The likelihood function for this situation is given by 


1 if to y 
L(x; 4,07) = (2n02)N/2 SP | = oe : | : 


On the one hand, as shown in subsection 31.5.1, the values of yw and o that 
maximise L in A are « = X and o? = s*, where X is the sample mean and 3? is 
the sample variance. On the other hand, to maximise L in the subspace S we set 
= fo, and the only remaining parameter is o7; the value of o? that maximises 
L is then easily found to be 


l- 


“9 Ee 


To retain, in due course, the standard notation for Student’s t-test, in this section 
we will denote the generalised likelihood ratio by / (rather than ft); it is thus 
given by 


L(x; 10, 02) 

L(x; X, 8?) 

[(2m/N) 3701 — Ho)°}-N/? exp(—N/2) _ 200 — XP 
~ [(2x/N) iC — X)]-%/? exp(—N/2) Soi — Ho)? 


Normally, our next step would be to find the sampling distribution of 2 under 
the assumption that Hp were true. It is more conventional, however, to work in 
terms of a related test statistic t, which was first devised by William Gossett, who 
wrote under the pen name of ‘Student’. 


A(x) = 


N/2 
. (31.112) 
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The sum of squares in the denominator of (31.112) may be put into the form 
DiGi — Ho)? = N(X— bo)? + Vii — XY. 


Thus, on dividing the numerator and denominator in (31.112) by 0 (x; — x)? and 
rearranging, the generalised likelihood ratio 4 can be written 


2 —N/2 
h=(1+——_— 
teed). 


where we have defined the new variable 
poor Sees (31.113) 
s/J/N-1 


Since t? is a monotonically decreasing function of J, the corresponding rejection 
region is t? > c, where c is a positive constant depending on the required 
significance level «. It is conventional, however, to use t itself as our test statistic, 
in which case our rejection region becomes two-tailed and is given by 


t < —teorit and t > torits (31.114) 


where tei, 1s the positive square root of the constant c. 

The definition (31.113) and the rejection region (31.114) form the basis of 
Student’s t-test. It only remains to determine the sampling distribution P(t|Hp). 
At the outset, it is worth noting that if we write the expression (31.113) for t 
in terms of the standard estimator 6 = ,/Ns2/(N — 1) of the standard deviation 
then we obtain 


en X — Ho 

6/./N 
If, in fact, we knew the true value of o and used it in this expression for t then 
it is clear from our discussion in section 31.3 that t would follow a Gaussian 
distribution with mean 0 and variance 1, i.e. t ~ N(0,1). When o is not known, 
however, we have to use our estimate @ in (31.115), with the result that t is 
no longer distributed as the standard Gaussian. As one might expect from 
the central limit theorem, however, the distribution of t does tend towards the 
standard Gaussian for large values of N. 

As noted earlier, the exact distribution of t, valid for any value of N, was first 
discovered by William Gossett. From (31.35), if the hypothesis Ho is true then the 
joint sampling distribution of X and s is given by 


2 7 2 
P(x, s|Ho) Ee CsN-? exp ( Ns ) exp | N(X Ho) ' 


(31.115) 


2 2 
i ae (31.116) 


where C is a normalisation constant. We can use this result to obtain the joint 
sampling distribution of s and t by demanding that 


P(x, s|Ho) dx ds = P(t, s|Ho) dt ds. 


1285 


STATISTICS 


Using (31.113) to substitute for x — wo in (31.116), and noting that dx = 


(s/./N — 1) dt, we find 


P(&, s|Ho) dx ds = As‘! ex Ne 1+ i dt ds 
xX, Xx ds = As -— ; 

° Pl 36 N-1 
where A is another normalisation constant. In order to obtain the sampling 
distribution of t alone, we must integrate P(t,s|Ho) with respect to s over its 
allowed range, from 0 to oo. Thus, the required distribution of t alone is given by 


iva) oO a Ns2 t? 
P(t\|Ho) = P(t,s|Ho)ds = A s* exp = 1+ Wo ds. 
0 0 . ~*74 (31.117) 


1/2 


To carry out this integration, we set y = s{1 + [t?/(N — 1)]}!/2, which on substi- 


tution into (31.117) yields 


2 —N/2 po Seg Ny? 
Pitt) =4(1+ 545) | y exp (-) dy. 


Since the integral over y does not depend on f, it is simply a constant. We thus 
find that that the sampling distribution of the variable t is 


1 2 —N/2 
Peace EN) (1+ 55) ‘ 
(31.118) 


[(N —T)x Vr (5(N — 1)) N-1 


where we have used the condition ‘ie P(t\|Ho) dt = 1 to determine the normali- 
sation constant (see exercise 31.18). 

The distribution (31.118) is called Student’s t-distribution with N —1 degrees of 
freedom. A plot of Student’s t-distribution is shown in figure 31.11 for various 
values of N. For comparison, we also plot the standard Gaussian distribution, 
to which the f-distribution tends for large N. As is clear from the figure, the 
t-distribution is symmetric about t = 0. In table 31.3 we list some critical points 
of the cumulative probability function C,(t) of the t-distribution, which is defined 
by 


t 
culty = [PU Hoar, 
where n = N —1 is the number of degrees of freedom. Clearly, C,,(t) is analogous 
to the cumulative probability function ®(z) of the Gaussian distribution, discussed 
in subsection 30.9.1. For comparison purposes, we also list the critical points of 
@®(z), which corresponds to the t-distribution for N = 00. 
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P(t|Ho) 


Figure 31.11 Student’s t-distribution for various values of N. The broken 
curve shows the standard Gaussian distribution for comparison. 


> Ten independent sample values x;, i= 1,2,...,10, are drawn at random from a Gaussian 
distribution with unknown mean yp and unknown standard deviation o. The sample values 
are as follows: 


2.22 2.56 107 0.24 0.18 0.95 0.73 -—0.79 2.09 1.81 
Test the null hypothesis Ho : 1 = 0 at the 10% significance level. 


For our null hypothesis, 4 = 0. Since for this sample X = 1.11, s = 1.01 and N = 10, it 
follows from (31.113) that 
t = —~— = 3333, 


s//N-1 
The rejection region for t is given by (31.114) where fo,i¢ is such that 
Cy-1(terit) =1—- «/2, 


and « is the required significance of the test. In our case « = 0.1 and N = 10, and from 
table 31.3 we find toi, = 1.83. Thus our rejection region for Hp at the 10% significance 
level is 


t < —1.83 and t > 1.83. 


For our sample t = 3.30 and so we can clearly reject the null hypothesis Ho : u = 0 at this 
level. < 


It is worth noting the connection between the t-test and the classical confidence 
interval on the mean yu. The central confidence interval on yp at the confidence 
level 1 — « is the set of values for which 


X= HU 
—terit < Sy apa 
Ss 
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06 #07 O08 09 0.950 0.975 0.990 0.995 0.999 


0.33 0.73 1.38 3.08 631 12.7 31.8 63.7 318.3 
0.29 0.62 106 189 292 430 697 9.93 22.3 
0.28 0.58 0.98 1.64 2.35 3.18 454 5.84 10.2 
0.27 0.57 0.94 1.53 2.13 2.78 3.75 460 7.17 


0.27 0.56 092 148 2.02 2.57 3.37 4.03 5.89 
0.27 055 O91 144 194 2.45 3.14 3.71 5.21 
0.26 0.55 090 142 190 2.37 3.00 3.50 4.79 
0.26 0.55 089 140 186 2.31 2.90 3.36 4.50 
0.26 0.54 O88 1.38 183 226 282 3.25 430 


0.26 0.54 088 1.37 181 2:23 2.76 317 4.14 
0.26 054 O88 1.36 180 2.20 2.72 3.11 4.03 
0.26 0.54 087 1.36 1.78 2.18 2.68 3.06 3.93 
0.26 0.54 O87 135 177 216 2.65 3.01 3.85 
0.26 054 087 135 176 2.15 2.62 2.98 3.79 


0.26 0.54 087 134 175 2.13 2.60 2.95 3.73 
0.26 054 O87 134 175 212 258 2.92 3.69 
0.26 0.53 086 1.33 174 211 2.57 2.90 3.65 
0.26 0.53 0.86 1.33 173 210 255 2.88 3.61 
0.26 0.53 086 1.33 173 209 254 2.86 3.58 


0.26 0.53 0.86 1.33 173 209 2.53 2.85 3.55 
0.26 0.53 086 1.32 L71 2.06 2.49 2.79 3.46 
0.26 0.53 085 131 4170 204 246 2.75 3.39 
0.26 0.53 085 1.30 168 202 242 2.70 3.31 
0.26 0.53 085 1.30 168 2.01 2.40 2.68 3.26 


0.25 0.53 085 1.29 1.66 1.98 2.37 2.63 3:17 
0.25 0.53 0.84 1.29 1.65 197 2.35 260 3.13 
0.25 0.52 084 1.28 1.65 1.96 2.33 2.58 3.09 


Table 31.3 The confidence limits t of the cumulative probability function 
C,(t) for Student’s t-distribution with n degrees of freedom. For example, 
Cs(0.92) = 0.8. The row n = o is also the corresponding result for the 
standard Gaussian distribution. 


where ferit Satisfies Cy—1(terit) = «/2. Thus the required confidence interval is 


ee Corit S 2 pee Corit S , 
N-1 N-1 


Hence, in the above example, the 90% classical central confidence interval on py 


1S 


0.49 < p< 1.73. 


The t-distribution may also be used to compare different samples from Gaussian 
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distributions. In particular, let us consider the case where we have two independent 
samples of sizes N; and N>, drawn respectively from Gaussian distributions with 
a common variance o? but with possibly different means p and py. On the basis 
of the samples, one wishes to distinguish between the hypotheses 


Hy 24 = 1b, 0< 0 <a and Ay sy # lb, 0<0’ <a. 


In other words, we wish to test the null hypothesis that the samples are drawn 
from populations having the same mean. Suppose that the measured sample 
means and standard deviations are X;, X2 and sj, s2 respectively. In an analogous 
way to that presented above, one may show that the generalised likelihood ratio 


can be written as 
2 —(Ni+N2)/2 
A=(14 , 
- ( "N,N =) 


In this case, the variable t is given by 


ae NUN 1/2 
r= See a) (31.119) 


where W = X1 — Xo, w = My — fb and 


— Nisy + Nos} ue 
Ni +N2.—2 


It is straightforward (albeit with complicated algebra) to show that the variable t 
in (31.119) follows Student’s t-distribution with Nj + Nz — 2 degrees of freedom, 
and so we may use an appropriate form of Student’s t-test to investigate the null 
hypothesis Ho : “1 = “2 (or equivalently Hp : @ = 0). As above, the t-test can be 
used to place a confidence interval on w = 44 — ww. 


> Suppose that two classes of students take the same mathematics examination and the 
following percentage marks are obtained: 


Class1: 66 62 34 55 77 80 55 60 69 47 50 
Class2: 64 90 76 56 81 72 70 


Assuming that the two sets of examinations marks are drawn from Gaussian distributions 
with a common variance, test the hypothesis Ho : ) = fy at the 5% significance level. Use 
your result to obtain the 95% classical central confidence interval on @ = py — fp. 


We begin by calculating the mean and standard deviation of each sample. The number of 
values in each sample is Nj = 11 and N, =7 respectively, and we find 


x,= 59.5, 3; = 12.8 and X2= aay = 10.3, 


leading to Ww = X; — X, = —13.2 and G = 12.6. Setting m = 0 in (31.119), we thus find 
t = —2.17. 
The rejection region for Ho is given by (31.114), where tot satisfies 


Cy, 4+Ny—2(terit) = 1 — 0/2, (31.120) 
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where « is the required significance level of the test. In our case we set « = 0.05, and from 
table 31.3 with n = 16 we find that tot = 2.12. The rejection region is therefore 


t<—2.12 and tS 212: 


Since t = —2.17 for our samples, we can reject the null hypothesis Ho : 1 = fo, although 
only by a small margin. (Indeed, it is easily shown that one cannot reject Ho at the 2% 
significance level). The 95% central confidence interval on w = 1; — fp is given by 


ais am)" oe eae 
W — Oteit | ———— < o <WwWt+6tei | ——— 5 
( NiNp *\ NING 


where ferit is given by (31.120). Thus, we find 
—26.1 <a < —0.28, 


which, as expected, does not (quite) contain w = 0. < 


In order to apply Student’s t-test in the above example, we had to make the 
assumption that the samples were drawn from Gaussian distributions possessing a 
common variance, which is clearly unjustified a priori. We can, however, perform 
another test on the data to investigate whether the additional hypothesis a7 = 0 
is reasonable; this test is discussed in the next subsection. If this additional test 
shows that the hypothesis a7 = o3 may be accepted (at some suitable significance 
level), then we may indeed use the analysis in the above example to infer that 
the null hypothesis Ho : 4; = f may be rejected at the 5% significance level. 
If, however, we find that the additional hypothesis 7 = co} must be rejected, 
then we can only infer from the above example that the hypothesis that the two 
samples were drawn from the same Gaussian distribution may be rejected at the 
5% significance level. 

Throughout the above discussion, we have assumed that samples are drawn 
from a Gaussian distribution. Although this is true for many random variables, 
in practice it is usually impossible to know a priori whether this is case. It can 
be shown, however, that Student’s t-test remains reasonably accurate even if the 
sampled distribution(s) differ considerably from a Gaussian. Indeed, for sampled 
distributions that differ only slightly from a Gaussian form, the accuracy of 
the test is remarkably good. Nevertheless, when applying the t-test, it is always 
important to remember that the assumption of a Gaussian parent population is 
central to the method. 


31.7.6 Fisher’s F-test 


Having concentrated on tests for the mean yw of a Gaussian distribution, we 
now consider tests for its standard deviation o. Before discussing Fisher’s F-test 
for comparing the standard deviations of two samples, we begin by considering 
the case when an independent sample x,,x2,...,xy is drawn from a Gaussian 
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0 a 10 b 20 30 40 
Figure 31.12 The sampling distribution P(u|Ho) for N = 10; this is a chi- 
squared distribution for N — 1 degrees of freedom. 


distribution with unknown yu and o, and we wish to distinguish between the two 
hypotheses 


Hy 10° =o, —0 << and H, :0° #0, —0 <<, 


where a is a given number. Here, the parameter space A is the half-plane 


—«0 < p< 0, 0 < a? < «, whereas the subspace S characterised by the null 
hypothesis Ho is the line o? = 0), —c0 << 00. 
The likelihood function for this situation is given by 
1 Dili — uw)? 
2 2) 
L(x; 1,0 ) = (nan? “*P ae : 


The maximum of L in A occurs at = X and o* = s*, whereas the maximum of 
Lin S is at 1 =X and o” =a. Thus, the generalised likelihood ratio is given by 
L(x; X, 02) u\N/2 
A(x) = TV" = (=) —1(u—N)], 
(x) L(x; X, 8?) N eae [ ) 
where we have introduced the variable 
Ns? Pes = xy 


= = oS". 31.121 
u a a ( ) 


An example of this distribution is plotted in figure 31.12 for N = 10. From 
the figure, we see that the rejection region A < Ac it corresponds to a two-tailed 
rejection region on u given by 


O<u<a and b<u<, 


where a and b are such that Agit(a) = Acrit(b), as shown in figure 31.12. In practice, 
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however, it is difficult to determine a and b for a given significance level «, so a 
slightly different rejection region, which we now describe, is usually adopted. 

The sampling distribution P(u|Ho) may be found straightforwardly from the 
sampling distribution of s given in (31.35). Let us first determine P(s?|Ho) by 
demanding that 


P(s\Ho) ds = P(s*|Ho) d(s’), 


from which we find 


ye: eel a5 ees eae _Ns’ (31.122) 
mn 2s 208 T (A(N — 1) Daag 


Thus, the sampling distribution of u = Ns*/a@ is given by 


1 
_ (N—3)/2 ak 
P(u|Ho) x0-DALp Cw = ) u exp ( 5u) : 
We note, in passing, that the distribution of u is precisely that of an (N — 1) th- 
order chi-squared variable (see subsection 30.9.4), ie. u ~ xy_1. Although it does 
not give quite the best test, one then takes the rejection region to be 


O<u<a and b<u<a, 


with a and b chosen such that the two tails have equal areas; the advantage of 
this choice is that tabulations of the chi-squared distribution make the size of this 
region relatively easy to estimate. Thus, for a given significance level «, we have 


ee) 


[Petey au = 0/2 and / P(u|Ho) du = «/2. 
0 b 


> Ten independent sample values x;, i= 1,2,...,10, are drawn at random from a Gaussian 
distribution with unknown mean pw and standard deviation o. The sample values are as 
follows: 


2.22 2.56 107 0.24 018 0.95 0.73 -—0.79 2.09 1.81 


Test the null hypothesis Hy : c? = 2 at the 10% significance level. 


For our null hypothesis a5 = 2. Since for this sample s = 1.01 and N = 10, from (31.121) 
we have u = 5.10. For « = 0.1 we find, either numerically or using table 31.2, that a = 3.33 
and b = 16.92. Thus, our rejection region is 


0<u < 3.33 and 16.92 <u<o. 


The value u = 5.10 from our sample does not lie in the rejection region, and so we cannot 
reject the null hypothesis Hy : 0? = 2. < 
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We now turn to Fisher’s F-test. Let us suppose that two independent samples 
of sizes N; and N> are drawn from Gaussian distributions with means and 
variances 1,07 and j,05 respectively, and we wish to distinguish between the 
two hypotheses 


Hy 107 = 03 and H, :0; #03. 
In this case, the generalised likelihood ratio is found to be 


(Ny +.N2) M422 [FIN — D/(N2 — 1)” 


NM?ONM? [1 NG 1)/(N> = 1)] (Ni +N2)/2? 


jo 
where F is given by the variance ratio 


Nisi/(N1 a 1) = uw 
Nys3/(No — 1) ~ 


(31.123) 


and s; and s» are the standard deviations of the two samples. On plotting 4 as a 
function of F, it is apparent that the rejection region 2 < Ay corresponds to a 
two-tailed test on F. Nevertheless, as will shall see below, by defining the fraction 
(31.123) appropriately, it is customary to make a one-tailed test on F. 

The distribution of F may be obtained in a reasonably straightforward manner 
by making use of the distribution of the sample variance s* given in (31.122). 
Under our null hypothesis Ho, the two Gaussian distributions share a common 
variance, which we denote by o?. Changing the variable in (31.122) from s? to u? 
we find that u? has the sampling distribution 


jeg (NTO 1 2\(N-3)/2 (N = wu? 
P(u’|Ho) = ( 752 ) rdw) ) exp | : 


Since uv? and v? are independent, their joint distribution is simply the product of 
their individual distributions and is given by 


Ny — 1)u? + (Ny — 1)v? 
Pu iHo)P(0 Fo) = Ata? )°-9/2(02)99 exp | mi wt ai |: 
oO 


where the constant A is given by 


(N, — 1) -D/2(Ny — 1)%2-D/2 
QUN1 +N2—2)/2 (Ni +N2—-2)P (5 —1))r (4(N> & 1)’ 


(31.124) 


Now, for fixed v we have u* = Fv? and d(u*) = v*dF. Thus, the joint sampling 
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distribution P(v?, F|Ho) is obtained by requiring that 


P(v?, F|Ho) d(v?) dF = P(u*|Ho)P(v?|Ho) d(u’) d(v”). 
(31.125) 


In order to find the distribution of F alone, we now integrate P(v?,F|Ho) with 
respect to v* from 0 to oo, from which we obtain 


P(F|Ho) 
= (= = oe F(N1-3)/2 (1 53 Ni— tr) —(Ni+N>—2)/2 
Ny—1 B (5(Mi — 1), 3(N2 — 1)) Ny—1 


(31.126) 


where B(5(Ni —1),5(N2—1)) is the beta function defined in the Appendix. 
P(F|Ho) is called the F-distribution (or occasionally the Fisher distribution) with 
(Ni — 1, No — 1) degrees of freedom. 


> Evaluate the integral [\° P(v*, F|Ho)d(v”) to obtain result (31.126). 


From (31.125), we have 


P(F|Hy) = AF™!-3)/2 [oper exp oe \ d(v?). 
0 Oo 


Making the substitution x = [(Ni — 1)F + (N2 — 1)]v?/(207), we obtain 


2e? (Nj +N 2—2)/, i re) ‘ 
P(F|Ho) = A | ————______ Be i) oa Melee ax 
oe [moo : 


IG? (N1+N2—2)/ 
7 lmonrEeco| FM OPL (GN + N22), 


where in the last line we have used the definition of the gamma function given in the 
Appendix. Using the further result (18.165), which expresses the beta function in terms of 
the gamma function, and the expression for A given in (31.124), we see that P(F|Hp) is 
indeed given by (31.126). < 


As it does not matter whether the ratio F given in (31.123) is defined as u*/v? 
or as v?/u?, it is conventional to put the larger sample variance on the top, so 
that F is always greater than or equal to unity. A large value of F indicates that 
the sample variances u? and v* are very different whereas a value of F close to 
unity means that they are very similar. Therefore, for a given significance «, it is 
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Cm (F) | m=1 2 3 4 5 6 iy 8 

ny =1 |} 161 200 216 225 230 234 237 239 

21) 18.5 190 192 192 193 193 194 19.4 

3 | 10.1 9.55 9.28 912 9.01 894 889 8.85 

4 | 7.71 6.94 659 639 626 6.16 609 6.04 

5 | 6.61 5.79 541 5.19 5.05 495 488 482 

6 | 5.99 5.14 476 453 439 428 421 415 

71 5.59 474 435 412 3.97 3.87 3.79 3.73 

8 | 5.32 446 407 3.84 3.69 3.58 3.50 3.44 

9 | 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 

10 | 4.96 410 3.71 348 3.33 3.22 3.14 3.07 

20 | 4.35 3.49 3.10 2.87 2.71 2.60 2.51 2.45 

30 | 4.17 3.32 2.92 2.69 253 242 2.33 2.27 

40 | 4.08 3.23 2.84 2.61 245 2.34 2.25 2.18 

50 | 4.03 3.18 2.79 2.56 240 2.29 2.20 2.13 

100 | 3.94 3.09 2.70 246 2.31 2.19 2.10 2.03 

oo | 3.84 3.00 2.60 2.37 2.21 2.10 201 1.94 
n=9 10 20 30 40 50 100 « 

ny =1 | 241 242 248 250 251 252 253 254 

2) 19.4 194 194 195 19.5 19.5 19.5 19.5 

3 | 8.81 8.79 8.66 8.62 859 858 8.55 8.53 

4 | 6.00 5.96 5.80 5.75 5.72 5.70 5.66 5.63 

5 | 4.77 474 456 450 446 444 441 437 

6 | 4.10 4.06 3.87 3.81 3.77 3.75 3.71 3.67 

7 | 3.68 3.64 3.44 3.38 3.34 3.32 3.27 3.23 

8 | 3.39 3.35 3.15 3.08 3.04 3.02 2.97 2.93 

9 | 3.18 3.14 294 286 2.83 2.80 2.76 2.71 

10 | 3.02 2.98 2.77 2.70 2.66 2.64 2.59 2.54 

20 | 2.39 2.35 212 2.04 199 1.97 191 1.84 

30 | 2.21 2.16 193 269 1.79 1.76 1.70 1.62 


40 | 2.12 2.08 184 1.74 169 166 1.59 1.51 
50 | 2.07 2.03 1.78 169 163 160 152 1.44 
100 | 1.97 193 168 157 1.52 148 1.39 1.28 
co | 1.88 183) 157 146 1.39 1.35 1.24 1.00 


Table 31.4 Values of F for which the cumulative probability function C;, ,,(F) 
of the F-distribution with (1,2) degrees of freedom has the value 0.95. For 
example, for nj = 10 and nz = 6, Ch, n, (4.06) = 0.95. 


customary to define the rejection region on F as F > Feit, where 


Forit 
Che / P(F|E) dF =, 
1 


and ny = N,; —1 and m = N,—1 are the numbers of degrees of freedom. 
Table 31.4 lists values of Foi, corresponding to the 5% significance level (i.e. 
a = 0.05) for various values of n; and np. 
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> Suppose that two classes of students take the same mathematics examination and the 
following percentage marks are obtained: 


Ckhesie @ @ 3b Ss) Wi tO sd @) @ 4 sv) 
Class2: 64 90 76 56 81 72 70 


Assuming that the two sets of examinations marks are drawn from Gaussian distributions, 
test the hypothesis Hy : oj = 0% at the 5% significance level. 


The variances of the two samples are sj = (12.8)? and s} = (10.3)? and the sample sizes 
are N; = 11 and N, = 7. Thus, we have 


2 N, st ‘ 2 Nos} 

u ag eae = 180.2 and v Ne ad 

where we have taken u? to be the larger value. Thus, F = u*/v? = 1.46 to two decimal 

places. Since the first sample contains eleven values and the second contains seven values, 

we take n, = 10 and n2 = 6. Consulting table 31.4, we see that, at the 5% significance 

level, Foit = 4.06. Since our value lies comfortably below this, we conclude that there is 

no statistical evidence for rejecting the hypothesis that the two samples were drawn from 
Gaussian distributions with a common variance. < 


= 123.8, 


It is also common to define the variable z = in F, the distribution of which 
can be found straightfowardly from (31.126). This is a useful change of variable 
since it can be shown that, for large values of mj and mp, the variable z is 
distributed approximately as a Gaussian with mean 5(nz! —n;!) and variance 
Lids 1 
x(ny +ny°). 


31.7.7 Goodness of fit in least-squares problems 


We conclude our discussion of hypothesis testing with an example of a goodness- 
of-fit test. In section 31.6, we discussed the use of the method of least squares in 
estimating the best-fit values of a set of parameters a in a given model y = f(x;a) 
for a data set (x;, yi), i = 1,2,...,N. We have not addressed, however, the question 
of whether the best-fit model y = f(x;4) does, in fact, provide a good fit to the 
data. In other words, we have not considered thus far how to verify that the 
functional form f of our assumed model is indeed correct. In the language of 
hypothesis testing, we wish to distinguish between the two hypotheses 


Hoy : model is correct and H, : model is incorrect. 


Given the vague nature of the alternative hypothesis Hj, we clearly cannot use 
the generalised likelihood-ratio test. Nevertheless, it is still possible to test the 
null hypothesis Ho at a given significance level «. 
The least-squares estimates of the parameters @,a@,...,@\, as discussed in 
section 31.6, are those values that minimise the quantity 
N 
Cla) = So bi - fxisalN- iby — f0xjsa)] = (y —8)'NLy — f. 


ij=1 
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In the last equality, we rewrote the expression in matrix notation by defining the 
column vector f with elements f; = f(x;;a). The value y7(a) at this minimum can 
be used as a statistic to test the null hypothesis Ho, as follows. The N quantities 

— f(xi;a) are Gaussian distributed. However, provided the function f(x;;a) is 
linear in the parameters a, the equations (31.98) that determine the least-squares 
estimate 4 constitute a set of M linear constraints on these N quantities. Thus, 
as discussed in subsection 30.15.2, the sampling distribution of the quantity y7(4) 
will be a chi-squared distribution with N—M degrees of freedom (d.o.f), which has 
the expectation value and variance 


E[?(@]=N—M and V{[/°(a)] = 2(N — M). 


Thus we would expect the value of (A) to lie typically in the range (NV — M) + 
/2(N — M). A value lying outside this range may suggest that the assumed model 
for the data is incorrect. A very small value of y7(4) is usually an indication that 
the model has too many free parameters and has ‘over-fitted’ the data. More 
commonly, the assumed model is simply incorrect, and this usually results in a 
value of y7(a) that is larger than expected. 

One can choose to perform either a one-tailed or a two-tailed test on the 
value of 7?(A). It is usual, for a given significance level «, to define the one-tailed 
rejection region to be y7(4) > k, where the constant k satisfies 


[ Puivan = (31.127) 
k 


and P(y2) is the PDF of the chi-squared distribution with n = N — M degrees of 
freedom (see subsection 30.9.4). 


> An experiment produces the following data sample pairs (xi, yi): 


mee igs) PPE PSL BM BD ah a eshil aR AN oth  AECNS) 
yi: 2.26 3.10 3.80 411 474 431 524 403 5.69 6.57 


where the x;-values are known exactly but each y;-value is measured only to an accuracy 
of o = 0.5. At the one-tailed 5% significance level, test the null hypothesis Ho that the 
underlying model for the data is a straight line y = mx +c. 


These data are the same as those investigated in section 31.6 and plotted in figure 31.9. As 
shown previously, the least squares estimates of the slope m and intercept c are given by 


m=1.11 and ¢ = 04. (31.128) 


Since the error on each y;-value is drawn independently from a Gaussian distribution with 
standard deviation o, we have 


N 


2 x f (xi5a) yi — mx; — € 
La)=>> ei) => —— i (31.129) 


i=1 i=l 


Inserting the values (31.128) into (31.129), we obtain 7?(m,¢) = 11.5. In our case, the 
number of data points is N = 10 and the number of fitted parameters is M = 2. Thus, the 
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number of degrees of freedom is n = N— M = 8. Setting n = 8 and « = 0.05 in (31.127) 
we find from table 31.2 that k = 15.51. Hence our rejection region is 


(mn, ) > 15.51. 


Since above we found 7?(/,¢) = 11.5, we cannot reject the null hypothesis that the 
underlying model for the data is a straight line y= mx-+c. < 


As mentioned above, our analysis is only valid if the function f(x;a) is linear 
in the parameters a. Nevertheless, it is so convenient that it is sometimes applied 
in non-linear cases, provided the non-linearity is not too severe. 


31.8 Exercises 


A group of students uses a pendulum experiment to measure g, the acceleration 
of free fall, and obtains the following values (in ms~7): 9.80, 9.84, 9.72, 9.74, 
9.87, 9.77, 9.28, 9.86, 9.81, 9.79, 9.82. What would you give as the best value and 
standard error for g as measured by the group? 

Measurements of a certain quantity gave the following values: 296, 316, 307, 278, 
312, 317, 314, 307, 313, 306, 320, 309. Within what limits would you say there is 
a 50% chance that the correct value lies? 

The following are the values obtained by a class of 14 students when measuring 
a physical quantity x: 53.8, 53.1, 56.9, 54.7, 58.2, 54.1, 56.4, 54.8, 57.3, 51.0, 55.1, 
55.0, 54.2, 56.6. 


(a) Display these results as a histogram and state what you would give as the 
best value for x. 

(b) Without calculation, estimate how much reliance could be placed upon your 
answer to (a). 

(c) Databooks give the value of x as 53.6 with negligible error. Are the data 
obtained by the students in conflict with this? 


Two physical quantities x and y are connected by the equation 
fa = 
ax!/2 4 b’ 

and measured pairs of values for x and y are as follows: 


xe 10 12 16 20 
y: 409 196 114 94 


Determine the best values for a and b by graphical means, and (either by hand 
or by using a built-in calculator routine) by a least-squares fit to an appropriate 
straight line. 
Measured quantities x and y are known to be connected by the formula 

ax 
4b’ 


where a and b are constants. Pairs of values obtained experimentally are 


y= 


x: 20 30 40 50 60 
y: 032 0.29 0.25 0.21 0.18 


Use these data to make best estimates of the values of y that would be obtained 


for (a) x = 7.0, and (b) x = —3.5. As measured by fractional error, which estimate 
is likely to be the more accurate? 
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Prove that the sample mean is the best linear unbiased estimator of the population 
mean y as follows. 


(a) If the real numbers aj, d2,...,a, satisfy the constraint Sy a; = C, where C 
is a given constant, show that )7/_, a? is minimised by a; = C/n for all i. 

(b) Consider the linear estimator ft = 5>}_, a;X;. Impose the conditions (i) that it 
is unbiased and (ii) that it is as efficient as possible. 


A population contains individuals of k types in equal proportions. A quantity X 
has mean 4; amongst individuals of type i and variance o?, which has the same 
value for all types. In order to estimate the mean of X over the whole population, 
two schemes are considered; each involves a total sample size of nk. In the first 
the sample is drawn randomly from the whole population, whilst in the second 
(stratified sampling) n individuals are randomly selected from each of the k types. 


Show that in both cases the estimate has expectation 


1 k 
w= edo 
i=1 


but that the variance of the first scheme exceeds that of the second by an amount 
i 
ken Sui — py). 
i=1 


Carry through the following proofs of statements made in subsections 31.5.2 and 
31.5.3 about the ML estimators 7 and 2. 


(a) Find the expectation values of the ML estimators t and A given, respectively, 
in (31.71) and (31.75). Hence verify equations (31.76), which show that, even 
though an ML estimator is unbiased, it does not follow that functions of it 
are also unbiased. 


(b) Show that E[#?] = (N+1)t?/N and hence prove that 7 is a minimum-variance 
estimator of t. 


Each of a series of experiments consists of a large, but unknown, number n 
(> 1) of trials in each of which the probability of success p is the same, but also 
unknown. In the ith experiment, i = 1,2,...,N, the total number of successes is 
x; (> 1). Determine the log-likelihood function. 


Using Stirling’s approximation to In(n — x), show that 


dIn(n—x) _ 1 


Spee ane 9 
dn 2(n — x) TAS) 


and hence evaluate 0("C,)/0n. 


By finding the (coupled) equations determining the ML estimators p and h, 
show that, to order n~', they must satisfy the simultaneous ‘arithmetic’ and 
‘geometric’ mean constraints 
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31.11 


31.12 


This exercise is intended to illustrate the dangers of applying formalised estimator 
techniques to distributions that are not well behaved in a statistical sense. 

The following are five sets of 10 values, all drawn from the same Cauchy 
distribution with parameter a. 


(i) 4.81 —1.24 1.30 —0.23 2.98 
—1.13 —8.32 2.62 —0.79 —2.85 

(ii) 0.07 1.54 0.38 —2.76 —8.82 
1.86 —4.75 4.81 1.14 —0.66 

(iii) 0.72 4.57 0.86 —3.86 0.30 
—2.00 2.65 —1744 —2.26 —8.83 


Ignoring the fact that the Cauchy distribution does not have a finite variance (or 
even a formal mean), show that a, the ML estimator of a, has to satisfy 


10 1 
s(@) = —a =5. O(*) 
> 14+x7/@ 


Using a programmable calculator, spreadsheet or computer, find the value of 
a that satisfies (*) for each of the data sets and compare it with the value 
a = 1.6 used to generate the data. Form an opinion regarding the variance of the 
estimator. 

Show further that if it is assumed that (E[4])° = E[4’], then E[@] = v3’°, where 
v2 is the second (central) moment of the distribution, which for the Cauchy 
distribution is infinite! 

According to a particular theory, two dimensionless quantities X and Y have 
equal values. Nine measurements of X gave values of 22, 11, 19, 19, 14, 27, 8, 
24 and 18, whilst seven measured values of Y were 11, 14, 17, 14, 19, 16 and 
14. Assuming that the measurements of both quantities are Gaussian distributed 
with a common variance, are they consistent with the theory? An alternative 
theory predicts that Y* = 7X; are the data consistent with this proposal? 

On a certain (testing) steeplechase course there are 12 fences to be jumped, and 
any horse that falls is not allowed to continue in the race. In a season of racing 
a total of 500 horses started the course and the following numbers fell at each 
fence: 


Fence: 1 2 3 4 5 6 7 8 9 10 11 12 
Falls: 62 75 49 29 33 25 30 17 #19 11 #15 = «12 


Use this data to determine the overall probability of a horse’s falling at a fence, 
and test the hypothesis that it is the same for all horses and fences as follows. 


(a) Draw up a table of the expected number of falls at each fence on the basis 
of the hypothesis. 
(b) Consider for each fence i the standardised variable 


estimated falls — actual falls 


7 ~ Standard deviation of estimated falls’ 
and use it in an appropriate ,? test. 

(c) Show that the data indicates that the odds against all fences being equally 
testing are about 40 to 1. Identify the fences that are significantly easier or 


harder than the average. 
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31.8 EXERCISES 


31.13 


31.14 


31.15 


A similar technique to that employed in exercise 31.12 can be used to test 
correlations between characteristics of sampled data. To illustrate this consider 
the following problem. 

During an investigation into possible links between mathematics and classical 
music, pupils at a school were asked whether they had preferences (a) between 
mathematics and english, and (b) between classical and pop music. The results 
are given below. 


Classical None Pop 


Mathematics 23 13 14 
None 17 17 36 
English 30 10 40 


By computing tables of expected numbers, based on the assumption that no 
correlations exist, and calculating the relevant values of y*, determine whether 
there is any evidence for 


(a) a link between academic and musical tastes, and 
(b) a claim that pupils either had preferences in both areas or had no preference. 


You will need to consider the appropriate value for the number of degrees of 
freedom to use when applying the 7? test. 

Three candidates X, Y and Z were standing for election to a vacant seat on 
their college’s Student Committee. The members of the electorate (current first- 
year students, consisting of 150 men and 105 women) were each allowed to 
cross out the name of the candidate they least wished to be elected, the other 
two candidates then being credited with one vote each. The following data are 
known. 


(a) X received 100 votes from men, whilst Y received 65 votes from women. 
(b) Z received five more votes from men than X received from women. 
(c) The total votes cast for X and Y were equal. 


Analyse this data in such a way that a 7? test can be used to determine whether 
voting was other than random (i) amongst men and (ii) amongst women. 
A particle detector consisting of a shielded scintillator is being tested by placing it 
near a particle source whose intensity can be controlled by the use of absorbers. 
It might register counts even in the absence of particles from the source because 
of the cosmic ray background. 

The number of counts n registered in a fixed time interval as a function of the 
source strength s is given in as: 


source strength s: 0 1 2 3 4 5 6 
counts n: 6 11 20 42 44 62 61 


At any given source strength, the number of counts is expected to be Poisson 
distributed with mean 


n=a+thbs, 


where a and b are constants. Analyse the data for a fit to this relationship and 
obtain the best values for a and b together with their standard errors. 


(a) How well is the cosmic ray background determined? 

(b) What is the value of the correlation coefficient between a and b? Is this 
consistent with what would happen if the cosmic ray background were 
imagined to be negligible? 

(c) Do the data fit the expected relationship well? Is there any evidence that the 
reported data ‘are too good a fit’? 
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31.16 


31.17 


31.18 


31.19 


31.20 


The function y(x) is known to be a quadratic function of x. The following table 
gives the measured values and uncorrelated standard errors of y measured at 
various values of x (in which there is negligible error): 


x 1 2 3 4 5 
y(x) 35405 20405 30405 65410 105+1.0 


Construct the response matrix R using as basis functions 1, x, x?. Calculate the 
matrix R'N~!R and show that its inverse, the covariance matrix V, has the form 


, (12592 —9708 1580 
=——|-9708 8413 —1461 
9184 \ 1580 —1461 269 


Use this matrix to find the best values, and their uncertainties, for the coefficients 
of the quadratic form for y(x). 

The following are the values and standard errors of a physical quantity f(6) 
measured at various values of @ (in which there is negligible error): 


0 0 1/6 1/4 n/3 
f(0) 3.72402 =1.98+0.1 0.06 + 0.1 2.05+ 0.1 


0 n/2 22/3 3n/4 T 
f(0)  —2.83+0.2 115+01 3.99+0.2 9.71404 


Theory suggests that f should be of the form a; + a2 cos 8 + a3 cos 20. Show that 
the normal equations for the coefficients a; are 


481.3a; + 158.44) — 43.8a; = 284.7, 
158.4a; + 218.847 + 62.1a; = —31.1, 
—43.8a, + 62.la, + 131.343 = 368.4. 


(a) If you have matrix inversion routines available on a computer, determine the 
best values and variances for the coefficients a; and the correlation between 
the coefficients a; and a. 

(b) If you have only a calculator available, solve for the values using a Gauss— 
Seidel iteration and start from the approximate solution a; = 2, a, = —2, 
a= 4. 


Prove that the expression given for the Student’s t-distribution in equation (31.118) 
is correctly normalised. 

Verify that the F-distribution P(F) given explicitly in equation (31.126) is symme- 
tric between the two data samples, i.e. that it retains the same form but with N; 
and N» interchanged, if F is replaced by F’ = F~!. Symbolically, if P’(F’) is the 
distribution of F’ and P(F) = n(F,N,, N2), then P’(F’) = n(F’, No, N41). 

It is claimed that the two following sets of values were obtained (a) by ran- 
domly drawing from a normal distribution that is N(0,1) and then (b) randomly 
assigning each reading to one of two sets A and B: 


Set A: —0.314 0.603 0.551 0.537 0.160 1.635 0.719 
0.610 0.482 —1.757 0.058 
Set B: —0.691 1.515 —1.642 —1.736 1.224 1.423 1.165 


Make tests, including t- and F-tests, to establish whether there is any evidence 
that either claims is, or both claims are, false. 
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31.11 


31.13 


31.9 Hints and answers 


Note that the reading of 9.28 m s~ is clearly in error, and should not be used in 
the calculation; 9.80 + 0.02 m s~?. 

(a) 55.1. (b) Note that two thirds of the readings lie within +2 of the mean and 
that 14 readings are being used. This gives a standard error in the mean ~ 0.6. 
(c) Student’s t has a value of about 2.5 for 13 d.o.f. (degrees of freedom), and 
therefore it is likely at the 3% significance level that the data are in conflict with 
the accepted value. 

Plot or calculate a least-squares fit of either x? versus x/y or xy versus y/x 
to obtain a = 1.19 and b = 3.4. (a) 0.16; (b) —0.27. Estimate (b) is the more 
accurate because, using the fact that y(—x) = —y(x), it is effectively obtained by 
interpolation rather than extrapolation. 

Recall that, because of the equal proportions of each type, the expected numbers 
of each type in the first scheme is n. Show that the variance of the estimator for 
the second scheme is o7/(kn). When calculating that for the first scheme, recall 
that x? = y? + o? and note that y? can be written as (4; — w+ pu). 

The log-likelihood function is 


N N N 
InL= 5 In "Cy, + SC x; Inp + (» - > “) In(1 — p); 
i=l i=1 i=l 


O("Cx) n % 
on ~In(—.) 2n(n — x) 


Ignore the second term on the RHS of the above to obtain 


N 
Soin (5) + NIn(1—p)=0. 
i=1 hs 


X = 180422, ¥ = 150411. ¢ = 4.92 giving ¢ = 1.21 for 14 d.of., and 
is significant only at the 75% level. Thus there is no significant disagreement 
between the data and the theory. For the second theory, only the mean values 
can be tested as Y* will not be Gaussian distributed. The difference in the means 
is Y* —n*X = 47+ 36 and is only significantly different from zero at the 82% 
level. Again the data is consistent with the proposed theory. 

Consider how many entries may be chosen freely in the table if all row and 
column totals are to match the observed values. It should be clear that for an 
m x n table the number of degrees of freedom is (m— 1)(n — 1). 


(a) In order to make the fractions expressing each preference or lack of pref- 
erence correct, the expected distribution, if there were no correlation, must 
be 


Classical None Pop 


Mathematics 17.5 10 22.5 
None 24.5 14 31.5 
English 28 16 36 


This gives a 7’ of 12.3 for four d.o.f., making it less than 2% likely, that no 
correlation exists. 


(b) The expected distribution, if there were no correlation, is 


Music preference No music preference 
Academic preference 104 26 
No academic preference 56 14 
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31.15 


3117 
31.19 


This gives a 7? of 1.2 for one d.o.f and no evidence for the claim. 


As the distribution at each value of s is Poisson, the best estimate of the 
measurement error is the square root of the number of counts, i.e../n(s). Linear 
regression gives a = 4.3+2.1 and b = 10.06 + 0.94. 


(a) The cosmic ray background must be present, since n(0) # 0 but its value of 
about 4 is uncertain to within a factor 2. 

(b) The correlation coefficient between a and b is —0.63. Yes; if a were reduced 
towards zero then b would have to be increased to compensate. 

(c) Yes, v7 = 4.9 for five d.o.f., which is almost exactly the ‘expected’ value, 
neither too good nor too bad. 


a, = 2.02 + 0.06, ay 2.99 + 0.09, a3 = 4.90 + 0.10; 12 = —0.60. 
Note that |dF| = |dF’/F’| and write 
N,—-1 N;—-1 (No — 1)F’ 
as 1+ : 
(No — DF’ (Ny — 1)F’ Ni—1 


1+ 
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Where the discussion of a topic runs over two consecutive pages, reference is made 
only to the first of these. For discussions spread over three or more pages the first and 
last page numbers are given; these references are usually to the major treatment of the 
corresponding topic. Isolated refences to a topic, including those appearing on consecutive 
pages, are listed individually. Some long topics are split, e.g. ‘Fourier transforms’ and 
‘Fourier transforms, examples’. The letter ‘n’ after a page number indicates that the topic 


is discussed in a footnote on the relevant page. 


A, B, one-dimensional irreps, 1090, 1102, 1109 
Abelian groups, 1044 
absolute convergence of series, 124, 831 
absolute derivative, 975-977 
acceleration vector, 335 
Adams method, 1024 
Adams—Moulton-Bashforth, predictor-corrector 
scheme, 1035 
addition rule for probabilities, 1125, 1130 
addition theorem for spherical harmonics 
Y;"(0, 6), 594 
adjoint, see Hermitian conjugate 
adjoint operators, 559-564 
adjustment of parameters, 795 
Airy integrals, 890-894 
Ai(z), 889 
algebra of 
complex numbers, 85 
functions in a vector space, 556 
matrices, 251 
power series, 134 
series, 131 
tensors, 938-941 
vectors, 213 
in a vector space, 242 
in component form, 218 
algebraic equations, numerical methods for, see 
numerical methods for algebraic equations 
alternating group, 1116 
alternating series test, 130 
ammonia molecule, symmetries of, 1042 


Ampeéere’s rule (law), 381, 408 
amplitude modulation of radio waves, 444 
amplitude-phase diagram, 914 
analytic (regular) functions, 826 
angle between two vectors, 221 
angular frequency, 693n 
in Fourier series, 419 
angular momentum, 933, 949 
and irreps, 1093 
of particle system, 950-952 
of particles, 338 
of solid body, 396, 951 
vector representation, 238 
angular momentum operator 
component, 658 
total, 659 
angular velocity, vector representation, 223, 238, 
353 
annihilation and creation operators, 667 
anti-Hermitian matrices, 271 
eigenvalues, 276-278 
imaginary nature, 277 
eigenvectors, 276-278 
orthogonality, 277 
anti-Stokes line, 905 
anticommutativity of vector or cross product, 
222 
antisymmetric functions, 416 
and Fourier series, 419 
and Fourier transforms, 445 
antisymmetric matrices, 270 
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general properties, see anti-Hermitian 
matrices 
antisymmetric tensors, 938, 941 
antithetic variates, in Monte Carlo methods, 
1014 
aperture function, 437 
approximately equal ~, definition, 132 
arbitrary parameters for ODE, 469 
arc length of 
plane curves, 73 
space curves, 341 
arccosech, arccosh, arccoth, arcsech, arcsinh, 
arctanh, see hyperbolic functions, inverses 
Archimedean upthrust, 396, 410 
area element in 
Cartesian coordinates, 188 
plane polars, 202 
area of 
circle, 71 
ellipse, 71, 207 
parallelogram, 223 
region, using multiple integrals, 191-193 
surfaces, 346 
as vector, 393-395, 408 
area, maximal enclosure, 779 
arg, argument of a complex number, 87 
Argand diagram, 84, 825 
argument, principle of the, 880 
arithmetic series, 117 
arithmetico-geometric series, 118 
arrays, see matrices 
associated Laguerre equation, 535, 621-624 
as example of Sturm—Liouville equation, 566, 
622 
natural interval, 567, 622 
associated Laguerre polynomials L¥"(x), 621 
as special case of confluent hypergeometric 
function, 634 
generating function, 623 
orthogonality, 622 
recurrence relations, 624 
Rodrigues’ formula, 622 
associated Legendre equation, 535, 587-593, 733, 
768 
general solution, 588 
as example of Sturm—Liouville equation, 566, 
590, 591 
general solution, 588 
natural interval, 567, 590, 591 
associated Legendre functions, 587-593 
of first kind P/"(x), 588, 733, 768 
generating function, 592 
normalisation, 590 
orthogonality, 590, 591 
recurrence relations, 592 
Rodrigues’ formula, 588 
of second kind Q’"(x), 588 
associative law for 
addition 


in a vector space of finite dimensionality, 
242 
in a vector space of infinite dimensionality, 
556 
of complex numbers, 86 
of matrices, 251 
of vectors, 213 
convolution, 447, 458 
group operations, 1043 
linear operators, 249 
multiplication 
of a matrix by a scalar, 251 
of a vector by a scalar, 214 
of complex numbers, 88 
of matrices, 253 
multiplication by a scalar 
in a vector space of finite dimensionality, 
242 
in a vector space of infinite dimensionality, 
556 
atomic orbitals, 1115 
d-states, 1106, 1108, 1114 
p-states, 1106 
s-states, 1144 
auto-correlation functions, 450 
automorphism, 1061 
auxiliary equation, 493 
repeated roots, 493 
average value, see mean value 
axial vectors, 949 


backward differences, 1019 
basis functions 
for linear least squares estimation, 1273 
in a vector space of infinite dimensionality, 
556 
of a representation, 1078 
change in, 1084, 1087, 1092 
basis vectors, 217, 243, 929, 1078 
derivatives, 965-968 
Christoffel symbol aie 965 
for particular irrep, 1106-1108, 1116 
linear dependence and independence, 217 
non-orthogonal, 245 
orthonormal, 244 
required properties, 217 
Bayes’ theorem, 1132 
Bernoulli equation, 477 
Bessel correction to variance estimate, 1248 
Bessel equation, 535, 602-607, 614, 615 
as example of Sturm—Liouville equation, 566 
natural interval, 608 
Bessel functions J,(x), 602-614, 729, 738 
as special case of confluent hypergeometric 
function, 634 
generating function, 613 
graph of, 606 
integral relationships, 610 
integral representation, 613-614 
orthogonality, 608-611 
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recurrence relations, 611-612 
second kind Y,(x), 607 
graph of, 607 


series, 604 
v =0, 606 
ySti/2, 605 


spherical j,(x), 615, 741 
zeros of, 729, 739 
Bessel inequality, 246, 559 
best unbiased estimator, 1232 
beta function, 638 
bias of estimator, 1231 
bilinear transformation, general, 110 
binary chopping, 990 
binomial coefficient "C;,, 27-30, 1135-1137 
elementary properties, 26 
identities, 27 
in Leibnitz’ theorem, 49 
negative n, 29 
non-integral n, 29 
binomial distribution Bin(n, p), 1168-1171 
and Gaussian distribution, 1185 
and Poisson distribution, 1174, 1177 
mean and variance, 1171 
MGF, 1170 
recurrence formula, 1169 
binomial expansion, 25-30, 140 
binormal to space curves, 342 
birthdays, different, 1134 
bivariate distributions, 1196-1207 
conditional, 1198 
continuous, 1197 
correlation, 1200-1207 
and independence, 1200 
matrix, 1203-1207 
positive and negative, 1200 
uncorrelated, 1200 
covariance, 1200-1207 
matrix, 1203 
expectation (mean), 1199 
independent, 1197, 1200 
marginal, 1198 
variance, 1200 
Boltzmann distribution, 171 
bonding in molecules, 1103, 1105-1108 
Born approximation, 149, 575 
Bose-Einstein statistics, 1138 
boundary conditions 
and characteristics, 700 
and Laplace equation, 764, 766 
for Green’s functions, 512, 514-516 
inhomogeneous, 515 
for ODE, 468, 470, 501 
for PDE, 681, 685-687 
for Sturm—Liouville equations, 564 
homogeneous and inhomogeneous, 685, 723, 
752, 754 
superposition solutions, 718-724 
types, 702-705 
bra vector (y|, 649 


brachistochrone problem, 784 
Bragg formula, 237 

branch cut, 835 

branch points, 835 

Bromwich integral, 884 

bulk modulus, 980 


calculus of residues, see zeros of a function of a 
complex variable and contour integration 
calculus of variations 
constrained variation, 785-787 
estimation of ODE eigenvalues, 790 
Euler-Lagrange equation, 776 
Fermat’s principle, 787 
Hamilton’s principle, 788 
higher-order derivatives, 782 
several dependent variables, 782 
several independent variables, 782 
soap films, 780 
variable end-points, 782-785 
calculus, elementary, 41-76 
cancellation law in a group, 1046 
canonical form, for second-order ODE, 516 
card drawing, see probability 
carrier frequency of radio waves, 445 
Cartesian coordinates, 217 
Cartesian tensors, 930-955 
algebra, 938-941 
contraction, 939 
definition, 935 
first-order, 932-935 
from scalar, 934 
general order, 935-954 
integral theorems, 954 
isotropic, 944-946 
physical applications, 934, 939-941, 950-954 
second-order, 935-954, 968 
symmetry and antisymmetry, 938 
tensor fields, 954 
zero-order, 932-935 
from vector, 935 
Cartesian tensors, particular 
conductivity, 952 
inertia, 951 
strain, 953 
stress, 953 
susceptibility, 952 
catenary, 781, 787 
Cauchy 
boundary conditions, 702 
distribution, 1152 
inequality, 853 
integrals, 851-853 
product, 131 
root test, 129, 831 
theorem, 849 
Cauchy—Riemann relations, 827-830, 849, 873, 
875 
in terms of z and z*, 829 
central differences, 1019 
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central limit theorem, 1194-1196 
central moments, see moments, central 
centre of a group, 1069 
centre of mass, 195 
of hemisphere, 195 
of semicircular lamina, 197 
centroid, 195 
of plane area, 195 
of plane curve, 197 
of triangle, 216 
CF, see complementary function 
chain rule for functions of 
one real variable, 46 
several real variables, 157 
change of basis, see similarity transformations 
change of variables 
and coordinate systems, 158-160 
in multiple integrals, 199-207 
evaluation of Gaussian integral, 202-204 
general properties, 206 
in RVD, 1150-1157 
character tables, 1093 
3m or 32 or C3, or S3, 1093, 1097, 1108, 1110, 
1117 
4mm or C4, or D4, 1102, 1106, 1108, 1113 
Ag, 1116 
Ds, 1116 
S4 or 432 or O, 1114, 1115 
43m or Tg, 1115 
construction of, 1100-1102 
quaternion, 1113 
characteristic equation, 280 
normal mode form, 319 
of recurrence relation, 499 
characteristic functions, see moment generating 
functions (MGFs) 
characteristics 
and boundary curves, 700 
multiple intersections, 700, 705 
and the existence of solutions, 699-705 
first-order equations, 699 
second-order equations, 703 
and equation type, 703 
characters, 1092-1096, 1100-1102 
and conjugacy classes, 1092, 1095 
counting irreps, 1095 
definition, 1092 
of product representation, 1104 
orthogonality properties, 1094, 1102 
summation rules, 1097 
charge (point), Dirac 6-function respresentation, 
441 
charged particle in electromagnetic fields, 370 
Chebyshev equation, 535, 595-602 
as example of Sturm—Liouville equation, 566, 
599 
general solution, 597 
natural interval, 567, 599 
polynomial solutions, 552 
Chebyshev functions, 595-602 


Chebyshev polynomials 
of first kind T,,(x), 596 
as special case of hypergeometric function, 
631 
generating function, 601 
graph of, 597 
normalisation, 600 
orthogonality, 599 
recurrence relations, 601 
Rodrigues’ formula, 599 
of second kind U,(x), 597 
generating function, 601 
graph of, 598 
normalisation, 600 
orthogonality, 599 
Rodrigues’ formula, 599 
chi-squared (7?) distribution, 1192 
and goodness of fit, 1297 
and likelihood-ratio test, 1283, 1292 
and multiple estimators, 1243 
percentage points tabulation, 1244 
test for correlation, 1301 
Cholesky separation, 313 
Christoffel symbol I*;;, 965-968 
from metric tensor, 966, 973 
circle 
area of, 71 
equation for, 16 
circle of convergence, 831 
Clairaut equation, 483 
classes and equivalence relations, 1064 
closure of a group, 1043 
closure property of eigenfunctions of an 
Hermitian operator, 563 
cofactor of a matrix element, 259 
column matrix, 250 
column vector, 250 
combinations (probability), 1133-1139 
common ratio in geometric series, 117 
commutation law for group elements, 1044 
commutative law for 
addition 
in a vector space of finite dimensionality, 
242 
in a vector space of infinite dimensionality, 
556 
of complex numbers, 86 
of matrices, 251 
of vectors, 213 
complex scalar or dot product, 222 
convolution, 447, 458 
inner product, 244 
multiplication 
of a vector by a scalar, 214 
of complex numbers, 88 
scalar or dot product, 220 
commutator 
of two matrices, 309 
of two operators, 653, 656 
comparison test, 125 
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complement, 1121 
probability for, 1125 
complementary equation, 490 
complementary error function, 640 
complementary function (CF), 491 
for ODE, 492 
partially known, 506 
repeated roots of auxiliary equation, 493 
completeness of 
basis vectors, 243 
eigenfunctions of an Hermitian operator, 560, 
563 
eigenvectors of a normal matrix, 275 
spherical harmonics Y/"(0, 6), 594 
completing the square 
as a means of integration, 66 
for quadratic equations, 35 
for quadratic forms, 1206 
to evaluate Gaussian integral, 436, 749 
complex conjugate 
z*, of complex number, 89-91, 829 
of a matrix, 256-258 
of scalar or dot product, 222 
properties of, 90 
complex exponential function, 92, 833 
complex Fourier series, 424 
complex integrals, 845-849, see also zeros of a 
function of a complex variable and contour 
integration 
Airy integrals, 890-894 
Cauchy integrals, 851-853 
Cauchy’s theorem, 849 
definition, 845 
Jordan’s lemma, 864 
Morera’s theorem, 851 
of z—!, 846 
principal value, 864 
residue theorem, 858-860 
WKB methods, 895-905 
complex logarithms, 99, 834 
principal value of, 100, 834 
complex numbers, 83-114 
addition and subtraction of, 85 
applications to differentiation and integration, 
101 
argument of, 87 
associativity of 
addition, 86 
multiplication, 88 
commutativity of 
addition, 86 
multiplication, 88 
complex conjugate of, see complex conjugate 
components of, 84 
de Moivre’s theorem, see de Moivre’s theorem 
division of, 91, 94 
from roots of polynomial equations, 83 
imaginary part of, 83 
modulus of, 87 
multiplication of, 88, 94 


as rotation in the Argand diagram, 88 
notation, 84 
polar representation of, 92-95 
real part of, 83 
trigonometric representation of, 93 
complex potentials, 871-876 
and fluid flow, 873 
equipotentials and field lines, 872 
for circular and elliptic cylinders, 876 
for parallel cylinders, 921 
for plates, 877-879, 921 
for strip, 921 
for wedges, 878 
under conformal transformations, 876-879 
complex power series, 133 
complex powers, 99 
complex variables, see functions of a complex 
variable and power series in a complex 
variable and complex integrals 
components 
of a complex number, 84 
of a vector, 217 
in a non-orthogonal basis, 234 
uniqueness, 243 
conditional (constrained) variation, 785-787 
conditional convergence, 124 
conditional distributions, 1198 
conditional probability, see probability, 
conditional 
cone 
surface area of, 74 
volume of, 75 
confidence interval, 1236 
confidence region, 1241 
confluence process, 634 
confluent hypergeometric equation, 535, 633 
as example of Sturm—Liouville equation, 566 
general solution, 633 
confluent hypergeometric functions, 633 
contiguous relations, 635 
integral representation, 634 
recurrence relations, 635 
special cases, 634 
conformal transformations (mappings), 839-879 
applications, 876-879 
examples, 842-844 
properties, 839-842 
Schwarz-—Christoffel transformation, 843 
congruence, 1065 
conic sections, 15 
eccentricity, 17 
parametric forms, 17 
standard forms, 16 
conjugacy classes, 1068-1070 
element in a class by itself, 1068 
conjugate roots of polynomial equations, 99 
connectivity of regions, 383 
conservative fields, 387-389 
necessary and sufficient conditions, 387-389 
potential (function), 389 
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consistency, of estimator, 1230 
constant coefficients in ODE, 492-503 
auxiliary equation, 493 
constants of integration, 62, 468 
constrained variation, 785-787 
constraints, stationary values under, see 
Lagrange undetermined multiplers 
continuity correction for discrete RV, 1186 
continuity equation, 404 
contour integration, 861-867, 887 
infinite integrals, 862-867 
inverse Laplace transforms, 884-887 
residue theorem, 858-867 
sinusoidal functions, 861 
summing series, 882 
contraction of tensors, 939 
contradiction, proof by, 32-34 
contravariant 
basis vectors, 961 
derivative, 965 
components of tensor, 956 
definition, 961 
control variates, in Monte Carlo methods, 1013 
convergence of infinite series, 831 
absolute, 124, 831 
complex power series, 133 
conditional, 124 
necessary condition, 125 
power series, 132 
under various manipulations, see power 
series, manipulation 
ratio test, 832 
rearrangement of terms, 124 
tests for convergence, 125-131 
alternating series test, 130 
comparison test, 125 
grouping terms, 129 
integral test, 128 
quotient test, 127 
ratio comparison test, 127 
ratio test (D’Alembert), 126, 132 
root test (Cauchy), 129, 831 
convergence of numerical iteration schemes, 
992-994 
convolution 
Fourier tranforms, see Fourier transforms, 
convolution 
Laplace tranforms, see Laplace transforms, 
convolution 
convolution theorem 
Fourier transforms, 448 
Laplace transforms, 457 
coordinate geometry, 15-18 
conic sections, 15 
straight line, 15 
coordinate systems, see Cartesian, curvilinear, 
cylindrical polar, plane polar and spherical 
polar coordinates 
coordinate transformations 
and integrals, see change of variables 


and matrices, see similarity transformations 
general, 960-965 
relative tensors, 963 
tensor transformations, 962 
weight, 964 
orthogonal, 932 
coplanar vectors, 225 
Cornu spiral, 914 
correlation functions, 449-451 
auto-correlation, 450 
cross-correlation, 449 
energy spectrum, 450 
Parseval’s theorem, 451 
Wiener-—Kinchin theorem, 450 
correlation matrix, of sample, 1229 
correlation of bivariate distributions, 1200-1207 
correlation of sample data, 1229 
correlation, chi-squared test, 1301 
correspondence principle in quantum mechanics, 
1215 
cosets and congruence, 1065 
cosh, hyperbolic cosine, 102, 833, see also 
hyperbolic functions 
cosine, cos(x) 
in terms of exponential functions, 102 
Maclaurin series for, 140 
orthogonality relations, 417 
counting irreps, see characters, counting irreps 
coupled pendulums, 329, 331 
covariance matrix 
of linear least squares estimators, 1274 
of sample, 1229 
covariance of bivariate distributions, 1200-1207 
covariance of sample data, 1229 
covariant 
basis vector, 961 
derivative, 965 
components of tensor, 956 
definition, 961 
derivative, 968 
of scalar, 971 
semi-colon notation, 969 
differentiation, 968-971 
CPF, see probability functions, cumulative 
Cramér-—Rao (Fisher’s) inequality, 1232, 1233 
Cramer determinant, 299 
Cramer’s rule, 299 
cross product, see vector product 
cross-correlation functions, 449 
crystal lattice, 148 
crystal point groups, 1082 
cube roots of unity, 98 
cube, rotational symmetries of, 1114 
cumulants, 1166 
curl of a vector field, 353 
as a determinant, 353 
as integral, 398, 400 
curl curl, 356 
in curvilinear coordinates, 368 
in cylindrical polars, 360 
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in spherical polars, 362 
Stoke’s theorem, 406-409 
tensor form, 974 


current-carrying wire, magnetic potential, 729 
curvature, 52-55 


circle of, 53 

of a function, 52 

of space curves, 342 
radius of, 53 


curves, see plane curves and space curves 
curvilinear coordinates, 364-369 


basis vectors, 364 

length and volume elements, 365 
scale factors, 364 

surfaces and curves, 364 

tensors, 955-977 

vector operators, 367-369 


cut plane, 865 

cycle notation for permutations, 1057 
cyclic groups, 1061, 1098 

cyclic relation for partial derivatives, 157 
cycloid, 370, 785 

cylinders, conducting, 874, 876 
cylindrical polar coordinates, 357-361 


é 
6 


ij> 


area element, 360 

basis vectors, 358 

Laplace equation, 728-731 
length element, 360 

vector operators, 357-361 
volume element, 360 


-function (Dirac), see Dirac 6-function 
6, Kronecker delta, tensor, see Kronecker 


delta, 6;;, of, tensor 


D’Alembert’s ratio test, 126, 832 


in convergence of power series, 132 


D’Alembert’s solution to wave equation, 694 
damped harmonic oscillators, 239 


and Parseval’s theorem, 451 


data modelling, maximum-likelihood, 1255 
de Broglie relation, 436, 709, 768 
de Moivre’s theorem, 95, 861 


d 
d 


d 


d 


applications, 95-99 
finding the nth roots of unity, 97 
solving polynomial equations, 98 
trigonometric identities, 95-97 
econvolution, 449 
efective matrices, 278, 311 
egeneracy 
breaking of, 1111-1113 
of normal modes, 1110 
egenerate (separable) kernel, 807 
egenerate eigenvalues, 275, 282 
egree 
of ODE, 468 
of polynomial equation, 2 
el V, see gradient operator (grad) 
el squared V? (Laplacian), 352, 676 
as integral, 400 


in curvilinear coordinates, 368 
in cylindrical polar coordinates, 360 
in polar coordinates, 725 
in spherical polar coordinates, 362, 741 
tensor form, 973 
delta function (Dirac), see Dirac 6-function 
dependent random variables, 1196-1205 
derivative, see also differentiation 
absolute, 975-977 
covariant, 968 
Fourier transform of, 444 
Laplace transform of, 455 
normal, 350 
of basis vectors, 336 
of composite vector expressions, 337 
of function of a complex variable, 825 
of function of a function, 46 
of hyperbolic functions, 106-109 
of products, 44-46, 48-50 
of quotients, 47 
of simple functions, 44 
of vectors, 334 
ordinary, first, second and nth, 42 
partial, see partial differentiation 
total, 154 
derivative method for second series solution of 
ODE, 545-548 
determinant form 
and ej jx, 942 
for curl, 353 
determinants, 259-263 
adding rows or columns, 262 
and singular matrices, 263 
as product of eigenvalues, 287 
evaluation 
using ej jx, 942 
using Laplace expansion, 259 
identical rows or columns, 262 
in terms of cofactors, 259 
interchanging two rows or two columns, 262 
Jacobian representation, 201, 205, 207 
notation, 259 
of Hermitian conjugate matrices, 262 
of order three, in components, 260 
of transpose matrices, 261 
product rule, 262 
properties, 261-263, 978 
relationship with rank, 267 
removing factors, 262 
secular, 280 
diagonal matrices, 268 
diagonalisation of matrices, 285—288 
normal matrices, 286 
properties of eigenvalues, 287 
simultaneous, 331 
diamond, unit cell, 234 
die throwing, see probability 
difference method for summation of series, 119 
difference schemes for differential equations, 
1020-1023, 1030-1032 


1311 


INDEX 


difference, finite, see finite differences 
differentiable 

function of a complex variable, 825-827 

function of a real variable, 42 
differentia! 

definition, 43 

exact and inexact, 155 

of vector, 338, 344 

total, 154 
differential equations, see ordinary differential 

equations and partial differential equations 

differential equations, particular 

associated Laguerre, 535, 566, 621-624 

associated Legendre, 535, 566, 587-593 

Bernoulli, 477 

Bessel, 535, 566, 602-607, 614 

Chebyshev, 535, 566, 595-602 

Clairaut, 483 

confluent hypergeometric, 535, 566, 633 

diffusion, 678, 695-698, 716, 723, 1032 

Euler, 504 

Euler-Lagrange, 776 

Helmholtz, 737-741 

Hermite, 535, 566, 624-628 

hypergeometric, 535, 566, 628-632 

Lagrange, 789 

Laguerre, 535, 566, 616-621 

Laplace, 679, 690, 717, 718, 1031 

Legendre, 534, 535, 566, 577-586 

Legendre linear, 503-505 

Poisson, 679, 744-746 

Schrédinger, 679, 741, 768, 795 

simple harmonic oscillator, 535, 566 

Sturm-Liouville, 790 

wave, 676, 689, 693-695, 714, 737, 790 
differential operators, see linear differential 

operator 

differentiation, see also derivative 

as gradient, 42 

as rate of change, 41 

chain rule, 46 

covariant, 968-971 

from first principles, 41-44 

implicit, 47 

logarithmic, 48 

notation, 43 

of Fourier series, 424 

of integrals, 178 

of power series, 135 

partial, see partial differentiation 

product rule, 44-46, 48-50 

quotient rule, 47 

theorems, 55-57 

using complex numbers, 101 
diffraction, see Fraunhofer diffraction 
diffusion equation, 678, 688, 695-698 

combination of variables, 696-698 

integral transforms, 747 

numerical methods, 1032 

separation of variables, 716 


simple solution, 696 
superposition, 723 
diffusion of solute, 678, 696, 747 
dihedral group, 1113, 1116 
dimension of irrep, 1088 
dimensionality of vector space, 243 
dipole matrix elements, 208, 1108, 1115 
dipole moments of molecules, 1077 
Dirac 6-function, 355, 405, 439-443 
and convolution, 447 
and Green’s functions, 511, 512 
as limit of various distributions, 443 
as sum of harmonic waves, 442 
definition, 439 
Fourier transform of, 443 
impulses, 441 
point charges, 441 
properties, 439 
reality of, 443 
relation to Fourier transforms, 442 
relation to Heaviside (unit step) function, 441 
three-dimensional, 441, 452 
Dirac notation, 648 
direct product, of groups, 1072 
direct sum ©, 1086 
direction cosines, 221 
Dirichlet boundary conditions, 702, 852n 
Green’s functions, 754, 756-765 
method of images, 758-765 
Dirichlet conditions, for Fourier series, 415 
disc, moment of inertia, 208 
discontinuous functions and Fourier series, 
420-422 
discrete Fourier transforms, 462 
disjoint events, see mutually exclusive events 
displacement kernel, 809 
distance from a 
line to a line, 231 
line to a plane, 232 
point to a line, 229 
point to a plane, 230 
distributive law for 
addition of matrix products, 254 
convolution, 447, 458 
inner product, 244 
linear operators, 249 
multiplication 
of a matrix by a scalar, 251 
of a vector by a complex scalar, 222 
of a vector by a scalar, 214 
multiplication by a scalar 
in a vector space of finite dimensionality, 
242 
in a vector space of infinite dimensionality, 
556 
scalar or dot product, 220 
vector or cross product, 222 
div, divergence of vector fields, 352 
as integral, 398 
in curvilinear coordinates, 367 
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in cylindrical polars, 360 of linear differential operators 

in spherical polars, 362 adjustment of parameters, 795 

tensor form, 972 definition, 555 
divergence theorem error in estimate of, 793 

for tensors, 954 estimation, 790-796 

for vectors, 401 higher eigenvalues, 793, 800 

in two dimensions, 384 simple harmonic oscillator, 555 

physical applications, 404 of linear operators, 272 

related theorems, 403 of normal matrices, 273-276 
division axiom in a group, 1046 of representative matrices, 1100 
division of complex numbers, 91 of unitary matrices, 278 
dominant term, in Stokes phenomenon, 904 under similarity transformation, 287 
dot product, see scalar product eigenvectors, 272-282 
double integrals, see multiple integrals characteristic equation, 280 
drumskin, see membrane definition, 272 
dual tensors, 949 determination, 280-282 


dummy variable, 61 normalisation condition, 273 


notation, 273 
€ijk> Levi-Civita symbol, tensor, 941-946 of anti-Hermitian matrices, see 
and determinant, 942 anti-Hermitian matrices 
identities, 943 f commuting matrices, 278 
isotropic, 945 general square matrices, 278 
vector products, 942 f Hermitian matrices, see Hermitian matrices 


|, Weight, 964 esiece linear operators, 272 
eX, see exponential function | matrices, 273-276 
E, two-dimensional irrep, 1090, 1102, 1108 cundany mniiece 278 


eccentricity, of conic sections, 17 . - c 
efficiency, of estimator, 1231 ee ee quagrancand 
eigenequation for differential operators, 554 Rinstein gelation 436, 709. 768 

more general form, 555, 571-573 elastic deformations, 953 


eigenfrequencies, 319 . electromagnetic fields 
estimation using Rayleigh—Ritz method, flux, 395 


eoooo °° 


a 


ei Raia Maxwell’s equations, 373, 408, 979 
8 a, electrostatic fields and potentials 
completeness for Hermitian operators, 560, h A selit sol 735 

563 charged split sphere, 


construction of a real set for an Hermitian conducting cylinder ie uniform field, 876 
operator, 563 conducting sphere in uniform field, 734 


definition, 555 from charge density, 745, 758 
from complex potential, 873 


normalisation for Hermitian operators, 562 : i 
infinite charged plate, 759, 877 


of integral equations, 817 


of simple harmonic oscillators, 555 infinite wedge with line charge, 878 

orthogonality for Hermitian operators, ininite charged wedge, 877 
561-563 of line charges, 761, 872 

eigenvalues, 272-282, see Hermitian operators semi-infinite charged plate, 877 

characteristic equation, 280 sphere with point charge, 764 

continuous and discrete, 650 ellipse 

definition, 272 area of, 71, 207, 385 

degenerate, 282 as section of quadratic surface, 292 

determination, 280-282 equation for, 16 

estimation for ODE, 790 ellipsoid, volume of, 207 

estimation using Rayleigh—Ritz method, elliptic PDE, 687, 690 
327-329 empty event 0, 1121 

notation, 273 end-points for variations 

of anti-Hermitian matrices, see contributions from, 782 
anti-Hermitian matrices fixed, 777 

of Fredholm equations, 808 variable, 782-785 

of general square matrices, 278 energy levels of 

of Hermitian matrices, see Hermitian matrices particle in a box, 768 

of integral equations, 808, 816 simple harmonic oscillator, 642 
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energy spectrum and Fourier transforms, 450, 
451 
entire functions, 832n 
envelopes, 173-175 
equations of, 174 
to a family of curves, 173 
epimorphism, 1061 
equilateral triangle, symmetries of, 1047, 1052, 
1081, 1110 
equivalence relations, 1064-1066, 1068 
and classes, 1064 
congruence, 1065-1067 
examples, 1070 
equivalence transformations, see similarity 
transformations 
equivalent representations, 1084-1086, 1099 
error function, erf(x), 640, 697, 748 
as special case of confluent hypergeometric 
function, 634 
error terms 
in Fourier series, 430 
in Taylor series, 139 
errors, first and second kind, 1280 
essential singularity, 838, 856 
estimation of eigenvalues 
linear differential operator, 792-795 
Rayleigh—Ritz method, 327-329 
estimators (statistics), 1229 
best unbiased, 1232 
bias, 1231 
central confidence interval, 1237 
confidence interval, 1236 
confidence limits, 1236 
confidence region, 1241 
consistency, 1230 
efficiency, 1231 
maximum-likelihood, 1256 
minimum-variance, 1232 
standard error, 1234 
Euler equation 
differential, 504, 522 
trigonometric, 93 
Euler method, numerical, 1021 
Euler-Lagrange equation, 776 
special cases, 777-781 
even functions, see symmetric functions 
events, 1120 
complement of, 1121 
empty 0, 1121 
intersection of N, 1120 
mutually exclusive, 1129 
statistically independent, 1129 
union of U, 1121 
exact differentials, 155 
exact equations, 472, 505 
condition for, 472 
non-linear, 519 
expectation values, see probability distributions, 
mean 
exponential distribution, 1190 


from Poisson, 1190 
MGF, 1191 
exponential function 
Maclaurin series for, 140 
of a complex variable, 92, 833 
relation with hyperbolic functions, 102 


F-distribution (Fisher), 1290-1296 
critical points table, 1295 
logarithmic form, 1296 
Fabry-Perot interferometer, 146 
factorial function, general, 636 
factorisation, of a polynomial equation, 7 
faithful representation, 1083, 1098 
Fermat’s principle, 787, 798 
Fermi—Dirac statistics, 1138 
Fibonacci series, 525 
field lines and complex potentials, 872 
fields 
conservative, 387-389 
scalar, 347 
tensor, 954 
vector, 347 
fields, electrostatic, see electrostatic fields and 
potentials 
fields, gravitational, see gravitational fields and 
potentials 
finite differences, 1019 
central, 1019 
for differential equations, 1020-1023 
forward and backward, 1019 
from Taylor series, 1019, 1026 
schemes for differential equations, 1030-1032 
finite groups, 1043 
first law of thermodynamics, 176 
first-order differential equations, see ordinary 
differential equations 
Fisher distribution, see F-distribution (Fisher) 
Fisher matrix, 1241, 1268 
Fisher’s inequality, 1232, 1233 
fluids 
Archimedean upthrust, 396, 410 
complex velocity potential, 873 
continuity equation, 404 
cylinder in uniform flow, 874 
flow, 873 
flux, 395, 875 
irrotational flow, 353 
sources and sinks, 404, 873 
stagnation points, 873 
velocity potential, 409, 679 
vortex flow, 408, 874 
forward differences, 1019 
Fourier cosine transforms, 446 
Fourier series, 415-432 
and separation of variables, 719-722, 724 
coefficients, 417-419, 425 
complex, 424 
differentiation, 424 
Dirichlet conditions, 415 
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discontinuous functions, 420-422 

error term, 430 

integration, 424 

non-periodic functions, 422-424 

orthogonality of terms, 417 
complex case, 425 

Parseval’s theorem, 426 

standard form, 417 

summation of series, 427 

symmetry considerations, 419 

uses, 415 

Fourier series, examples 

square-wave, 418 

x, 424, 425 

x, 422 

x3, 424 


Fourier sine transforms, 445 
Fourier transforms, 433-453 


as generalisation of Fourier series, 433-435 

convolution, 446-449 
and the Dirac 6-function, 447 
associativity, commutativity, distributivity, 
447 
definition, 447 
resolution function, 446 

convolution theorem, 448 

correlation functions, 449-451 

cosine transforms, 446 

deconvolution, 449 

definition, 435 

discrete, 462 

evaluation using convolution theorem, 448 

for integral equations, 809-812 

for PDE, 749-751 

Fourier-related (conjugate) variables, 436 

in higher dimensions, 451-453 

inverse, definition, 435 

odd and even functions, 445 

Parseval’s theorem, 450 

properties: differentiation, exponential 
multiplication, integration, scaling, 
translation, 444 

relation to Dirac 6-function, 442 

sine transforms, 445 

Fourier transforms, examples 

convolution, 448 

damped harmonic oscillator, 451 

Dirac 6-function, 443 

exponential decay function, 435 

Gaussian (normal) distribution, 435 

rectangular distribution, 442 

spherically symmetric functions, 452 

two narrow slits, 448 

two wide slits, 438, 448 


Fourier’s inversion theorem, 435 
Fraunhofer diffraction, 437-439 


diffraction grating, 461 

two narrow slits, 448 

two wide slits, 438, 448 
Fredholm integral equations, 805 


eigenvalues, 808 
operator form, 806 
with separable kernel, 807 
Fredholm theory, 815 
Frenet—Serret formulae, 343 
Fresnel integrals, 913 
Frobenius series, 539 
Fuch’s theorem, 539 
function of a matrix, 255 
functional, 776 
functions of a complex variable, 825-839, 
853-858 
analyticity, 826 
behaviour at infinity, 839 
branch points, 835 
Cauchy integrals, 851-853 
Cauchy—Riemann relations, 827-830 
conformal transformations, 839-844 
derivative, 825 
differentiation, 825-830 
identity theorem, 854 
Laplace equation, 829, 871 
Laurent expansion, 855-858 
multivalued and branch cuts, 835-837, 885 
particular functions, 832-835 
poles, 837 
power series, 830-832 
real and imaginary parts, 825, 830 
singularities, 826, 837-839 
Taylor expansion, 853-855 
zeros, 839, 879-882 
functions of one real variable 
decomposition into even and odd functions, 
416 
differentiation of, 41-50 
Fourier series, see Fourier series 
integration of, 59-72 
limits, see limits 
maxima and minima of, 50-52 
stationary values of, 50-52 
Taylor series, see Taylor series 
functions of several real variables 
chain rule, 157 
differentiation of, 151-179 
integration of, see multiple integrals, 
evaluation 
maxima and minima, 162-167 
points of inflection, 162-167 
rates of change, 153-155 
saddle points, 162-167 
stationary values, 162-167 
Taylor series, 160-162 
fundamental solution, 757 
fundamental theorem of 
algebra, 83, 85, 868 
calculus, 61 
complex numbers, see de Moivre’s theorem 


gamma distribution, 1153, 1191 
gamma function 
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as general factorial function, 636 
definition and properties, 636 
graph of, 637 
Gauss’s theorem, 765 
Gauss-Seidel iteration, 996-998 
Gaussian (normal) distribution N(u,07), 
1179-1189 
and binomial distribution, 1185 
and central limit theorem, 1195 
and Poisson distribution, 1187 
continuity correction, 1186 
CPF, 1018, 1181 
tabulation, 1182 
Fourier transform, 435 
integration with infinite limits, 202-204 
mean and variance, 1180-1184 
MGF, 1185, 1188 
multiple, 1188 
multivariate, 1209 
random number generation, 1018 
sigma limits, 1183 
standard variable, 1180 
Gaussian elimination with interchange, 995 
Gaussian integration, 1005-1009 
points and weights, 1008, 1010 
general tensors 
algebra, 938-941 
contraction, 939 
contravariant, 961 
covariant, 961 
dual, 949 
metric, 957-960 
physical applications, 957-960, 976 
pseudotensors, 964 
tensor densities, 964 
generalised likelihood ratio, 1282 
generating functions 
associated Laguerre polynomials, 623 
associated Legendre polynomials, 592 
Bessel functions, 613 
Chebyshev polynomials, 601 
Hermite polynomials, 627 
Laguerre polynomials, 620 
Legendre polynomials, 584-586 
generating functions in probability, 1157-1167, 
see also moment generating functions and 
probability generating functions 
geodesics, 797, 976, 982 
geometric distribution, 1159, 1172 
geometric series, 117 
Gibbs’ free energy, 178 
Gibbs’ phenonmenon, 421 
gradient of a function of 
one variable, 42 
several real variables, 153-155 
gradient of scalar, 348-352 
tensor form, 972 
gradient of vector, 937, 969 
gradient operator (grad), 348 
as integral, 398 


in curvilinear coordinates, 367 
in cylindrical polars, 360 
in spherical polars, 362 
tensor form, 972 
Gram-Schmidt orthogonalisation of 
eigenfunctions of Hermitian operators, 562 
eigenvectors of 
Hermitian matrices, 277 
normal matrices, 275 
functions in a Hilbert space, 557 
gravitational fields and potentials 
Laplace equation, 679 
Newton’s law, 339 
Poisson equation, 679, 744 
uniform disc, 771 
uniform ring, 742 
Green’s functions, 568-571, 751-767 
and boundary conditions, 512, 514 
and Dirac 6-function, 511 
and partial differential operators, 753 
and Wronskian, 527 
diffusion equation, 749 
Dirichlet problems, 756-765 
for ODE, 185, 511-516 
Neumann problems, 765-767 
particular integrals from, 514 
Poisson’s equation, 755 
Green’s theorems 
applications, 706, 754, 849 
in a plane, 384-387, 407 
in three dimensions, 402 
ground-state energy 
harmonic oscillator, 796 
hydrogen atom, 800 
group multiplication tables, 1050 
order three, 1062 
order four, 1050, 1052, 1061 
order five, 1062 
order six, 1055, 1061 
grouping terms as a test for convergence, 129 
groups 
Abelian, 1044 
associative law, 1043 
cancellation law, 1046 
centre, 1069 
closure, 1043 
cyclic, 1061 
definition, 1043-1046 
direct product, 1072 
division axiom, 1046 
elements, 1043 
order, 1047 
finite, 1043 
identity element, 1043-1046 
inverse, 1043, 1046 
isomorphic, 1051 
mappings between, 1059-1061 
homomorphic, 1059-1061 
image, 1059 
isomorphic, 1059 
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nomenclature, 1102 
non-Abelian, 1052-1056 
order, 1043, 1081, 1082, 1094, 1097, 1100 
permutation law, 1047 
subgroups, see subgroups 
groups, examples 
1 and —1 under multiplication, 1043 
alternating, 1116 
complex numbers e””, 1048 
functions, 1055 
general linear, 1073 
integers under addition, 1043 
integers under multiplication (mod N), 
1049-1051 
matrices, 1054 
permutations, 1056-1058 
quaternion, 1073 
rotation matrices, 1048 
symmetries of a square, 1100 
symmetries of an equilateral triangle, 1047 


H,,(x), see Hermite polynomials 
Hamilton’s principle, 788 
Hamiltonian, 796 
Hankel functions H!))(x), H?(x), 607 
Hankel transforms, 459 
harmonic oscillators 

damped, 239, 451 

ground-state energy, 796 

Schrédinger equation, 796 

simple, see simple harmonic oscillator 
heat flow 

diffusion equation, 678, 696, 723 

in bar, 723, 749, 770 

in thin sheet, 698 
Heaviside function, 441 

relation to Dirac 6-function, 441 
Heisenberg’s uncertainty principle, 435-437 
Helmholtz equation, 737-741 

cylindrical polars, 740 

plane polars, 738 

spherical polars, 740-741 
Helmholtz potential, 177 
hemisphere, centre of mass and centroid, 195 
Hermite equation, 535, 624-628 

as example of Sturm—Liouville equation, 566 

natural interval, 567 
Hermite polynomials H,(x), 625 

as special case of confluent hypergeometric 

function, 634 

generating function, 627 

graph of, 625 

normalisation, 626 

orthogonality, 626 

recurrence relations, 628 

Rodrigues’ formula, 626 
Hermitian conjugate, 256-258 

and inner product, 258 

product rule, 257 
Hermitian forms, 288-292 


positive definite and semi-definite, 290 
stationary properties of eigenvectors, 290 
Hermitian kernel, 816 
Hermitian matrices, 271 
eigenvalues, 276-278 
reality, 276 
eigenvectors, 276-278 
orthogonality, 277 
Hermitian operators, 559-564 
and physical variables, 650 
boundary condition for simple harmonic 
oscillators, 560 
eigenfunctions 
completeness, 560, 563 
orthogonality, 561-563 
eigenvalues 
reality, 561 
Green’s functions, 568-571 
importance of, 555, 560 
in Sturm-—Liouville equations, 564 
properties, 561-564 
superposition methods, 568-571 
higher-order differential equations, see ordinary 
differential equations 
Hilbert spaces, 557-559 
hit or miss, in Monte Carlo methods, 1014 
homogeneous 
boundary conditions, see boundary 
conditions, homogeneous and 
inhomogeneous 
differential equations, 490 
dimensionally consistent, 475, 521 
simultaneous linear equations, 293 
homomorphism, 1059-1061 
kernel of, 1060 
representation as, 1083 
Hooke’s law, 953 
hydrogen atom 
s-states, 1144 
electron wavefunction, 208 
ground-state energy, 800 
hydrogen molecule, symmetries of, 1041 
hyperbola 
as section of quadratic surface, 292 
equation for, 16 
hyperbolic functions, 102-109, 833 
calculus of, 106-109 
definitions, 102, 833 
graphs, 102 
identities, 104 
in equations, 105 
inverses, 105 
graphs, 106 
trigonometric analogies, 102-104 
hyperbolic PDE, 687, 690 
hypergeometric distribution, 1173 
mean and variance, 1173 
hypergeometric equation, 535, 628-632 
as example of Sturm—Liouville equation, 566, 
567 
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general solution, 630 
natural interval, 567 
hypergeometric functions, 628-632 
contiguous relations, 632 
integral representation, 631 
recurrence relations, 632 
special cases, 630 
hypothesis testing, 1277-1298 
errors, first and second kind, 1280 
generalised likelihood ratio, 1282 
generalised likelihood-ratio test, 1281 
goodness of fit, 1296 
Neyman-—Pearson test, 1280 
null, 1278 
power, 1280 
rejection region, 1279 
simple or composite, 1278 
statistical tests, 1278 
test statistic, 1278 


i, j, k (unit vectors), 219 

i, square root of —1, 84 

identity element of a group, 1043-1046 
uniqueness, 1043, 1045 

identity matrices, 254, 255 

identity operator, 249 

images, method of, see method of images 


imaginary part or term of a complex number, 83 
importance sampling, in Monte Carlo methods, 


1012 

improper 

integrals, 70 

rotations, 946-948 
impulses, 6-function respresentation, 441 
incomplete gamma function, 639 
independent random variables, 1156, 1200 
index of a subgroup, 1066 
indices, of regular singular points, 540 
indicial equation, 540 

distinct roots with non-integral difference, 

540-542 

repeated roots, 542, 546, 547 

roots differ by integer, 542, 546 
induction, proof by, 31 
inequalities 

amongst integrals, 72 

Bessel, 246, 559 

Schwarz, 246, 559 

triangle, 246, 559 
inertia, see also moments of inertia 

moments and products, 951 

tensor, 951 
inexact differentials, 155 
inexact equation, 473 
infinite integrals, 70 

contour integration, 862-867 
infinite series, see series 
inflection 

general points of, 52 

stationary points of, 50-52 


inhomogeneous 
boundary conditions, see boundary 
conditions, homogeneous and 
inhomogeneous 
differential equations, 490 
simultaneous linear equations, 293 


inner product in a vector space, see also scalar 


product 
of finite dimensionality, 244 
and Hermitian conjugate, 258 
commutativity, 244 
distributivity over addition, 244 
of infinite dimensionality, 557 
integral equations 
eigenfunctions, 817 
eigenvalues, 808, 816 
Fredholm, 805 
from differential equations, 803 
homogeneous, 805 
linear, 804 
first kind, 805 
second kind, 805 
nomenclature, 804 
singular, 805 
Volterra, 805 
integral equations, methods for 
differentiation, 812 
Fredholm theory, 815 
integral transforms, 809-812 
Fourier, 809-812 
Laplace, 810 
Neumann series, 813-815 
Schmidt—Hilbert theory, 816-819 
separable (degenerate) kernels, 807 
integral functions, 832n 
integral test for convergence of series, 128 


integral transforms, see also Fourier transforms 


and Laplace transforms 

general form, 459 

Hankel transforms, 459 

Mellin transforms, 459 
integrals, see also integration 

complex, see complex integrals 

definite, 59 

double, see multiple integrals 

Fourier transform of, 444 

improper, 70 

indefinite, 62 

inequalities, 72, 559 


infinite, 70 
Laplace transform of, 456 
limits 
containing variables, 188 
fixed, 59 


variable, 61 
line, see line integrals 
multiple, see multiple integrals 
non-zero, 1104 
of vectors, 339 
properties, 60 


1318 


INDEX 


triple, see multiple integrals 
undefined, 59 
integrand, 59 
integrating factor (IF), 506 
first-order ODE, 473-475 
integration, see also integrals 
applications, 72-76 
finding the length of a curve, 73 
mean value of a function, 72 
surfaces of revolution, 74 
volumes of revolution, 75 
as area under a curve, 59 
as the inverse of differentiation, 61 
formal definition, 59 
from first principles, 59 
in plane polar coordinates, 70 
logarithmic, 64 
multiple, see multiple integrals 
multivalued functions, 865-867, 885 
of Fourier series, 424 
of functions of several real variables, see 
multiple integrals 
of hyperbolic functions, 106-109 
of power series, 135 
of simple functions, 62 
of singular functions, 70 
of sinusoidal functions, 63, 861 
integration constant, 62 
integration, methods for 
by inspection, 62 
by parts, 67-69 
by substitution, 65-67 
t substitution, 65 
change of variables, see change of variables 
completing the square, 66 
contour, see contour integration 
Gaussian, 1005-1009 
numerical, 1000-1009 
partial fractions, 64 
reduction formulae, 69 
stationary phase, 912-920 
steepest descents, 908-912 
trigonometric expansions, 63 
using complex numbers, 101 
intersection M, probability for, 1120, 1128 
intrinsic derivative, see absolute derivative 
invariant tensors, see isotropic tensors 
inverse hyperbolic functions, 105 
inverse integral transforms 
Fourier, 435 
Laplace, 454, 884-887 
uniqueness, 454 
inverse matrices, 263-266 
elements, 264 


in solution of simultaneous linear equations, 


295 
product rule, 266 
properties, 265 
inverse of a linear operator, 249 
inverse of a product in a group, 1046 


inverse of element in a group 


uniqueness, 1043, 1046 


inversion theorem, Fourier’s, 435 
inversions as 


improper rotations, 946 
symmetry operations, 1041 


irregular singular points, 534 
irreps, 1087 


counting, 1095 

dimension n;,, 1097 

direct sum ©, 1086 

identity A;, 1100, 1104 

n-dimensional, 1088, 1089, 1102 
number in a representation, 1087, 1095 
one-dimensional, 1088, 1089, 1093, 1099, 1102 
orthogonality theorem, 1090-1092 
projection operators for, 1107 
reduction to, 1096 

summation rules for, 1097-1099 


irrotational vectors, 353 
isobaric ODE, 476 


non-linear, 521 


isoclines, method of, 1028, 1037 
isomorphic groups, 1051-1056, 1058, 1059 
isomorphism (mapping), 1060 

isotope decay, 484, 525 

isotropic (invariant) tensors, 944-946, 953 
iteration schemes 


Jy 


convergence of, 992-994 

for algebraic equations, 986-994 
for differential equations, 1025 
for integral equations, 813-816 
Gauss-Seidel, 996-998 

order of convergence, 993 


(x), see Bessel functions 


j, square root of —1, 84 
Jz(x), see spherical Bessel functions 
Jacobians 


analogy with derivatives, 207 
and change of variables, 206 
definition in 
two dimensions, 201 
three dimensions, 205 
general properties, 206 
in terms of a determinant, 201, 205, 207 


joint distributions, see bivariate distributions 


and multivariate distributions 


Jordan’s lemma, 864 


kernel of a homomorphism, 1060, 1063 
kernel of an integral transform, 459 
kernel of integral equations 


displacement, 809 

Hermitian, 816 

of form exp(—ixz), 810-812 

of linear integral equations, 804 
resolvent, 814, 815 

separable (degenerate), 807 


ket vector |y), 648 
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kinetic energy of oscillating system, 316 
Klein—Gordon equation, 711, 772 
Kronecker delta 6;; and orthogonality, 244 


Kronecker delta, 6;;, 5, tensor, 928, 941-946, 
956, 962 
identities, 943 
isotropic, 945 
vector products, 942 
Kummer function, 633 
kurtosis, 1150, 1227 


L(x), see Laguerre polynomials 
Li"(x), see associated Laguerre polynomials 
LHoOpital’s rule, 142-144 
Lagrange equations, 789 
and energy conservation, 797 
Lagrange undetermined multipliers, 167-173 
and ODE eigenvalue estimation, 792 
application to stationary properties of the 
eigenvectors of quadratic and Hermitian 
forms, 290 
for functions of more than two variables, 
169-173 
in deriving the Boltzmann distribution, 
171-173 
integral constraints, 785 
with several constraints, 169-173 
Lagrange’s identity, 226 
Lagrange’s theorem, 1065 
and the order of a subgroup, 1062 
and the order of an element, 1062 
Lagrangian, 789, 797 
Laguerre equation, 535, 616-621 
as example of Sturm—Liouville equation, 566, 
619 
natural interval, 567, 619 
Laguerre polynomials L;,(x), 617 
as special case of confluent hypergeometric 
function, 634 
generating function, 620 
graph of, 617 
normalisation, 619 
orthogonality, 619 
recurrence relations, 620 
Rodrigues’ formula, 618 
Lamé constants, 953 
lamina: mass, centre of mass and centroid, 
193-195 
Laplace equation, 679 
expansion methods, 741-744 
in two dimensions, 688, 690, 717, 718 
and analytic functions, 829 
and conformal transformations, 876-879 
numerical method for, 1031, 1038 
plane polars, 725-727 
separated variables, 717 
in three dimensions 
cylindrical polars, 728-731 
spherical polars, 731-737 
uniqueness of solution, 741 


with specified boundary values, 764, 766 
Laplace expansion, 259 
Laplace transforms, 453-459, 884 
convolution 
associativity, commutativity, distributivity, 
458 
definition, 457 
convolution theorem, 457 
definition, 453 
for ODE with constant coefficients, 501-503 
for PDE, 747-748 
inverse, 454, 884-887 
uniqueness, 454 
properties: translation, exponential 
multiplication, etc., 456 
table for common functions, 455 
Laplace transforms, examples 
constant, 453 
derivatives, 455 
exponential function, 453 
integrals, 456 
polynomial, 453 
Laplacian, see del squared V* (Laplacian) 
Laurent expansion, 855-858 
analytic and principal parts, 855 
region of convergence, 855 
least squares, method of, 1271-1277 
basis functions, 1273 
linear, 1272 
non-linear, 1276 
response matrix, 1273 
Legendre equation, 534, 535, 577-586 
as example of Sturm—Liouville equation, 566, 
583 
associated, see associated Legendre equation 
general solution, 578, 580 
natural interval, 567, 583 
Legendre functions P(x), 577-586 
associated Legendre functions, 768 
of second kind Q;(x), 579 
graph of, 580 
Legendre linear equation, 503 
Legendre polynomials P,(x), 578 
as special case of hypergeometric function, 
631 
associated Legendre functions, 733 
generating function, 584-586 
graph of, 579 
in Gaussian integration, 1006 
normalisation, 578, 582 
orthogonality, 583, 735 
recurrence relations, 585, 586 
Rodrigues’ formula, 581 
Leibnitz’ rule for differentiation of integrals, 178 
Leibnitz’ theorem, 48-50 
length of 
a vector, 218 
plane curves, 73, 341 
space curves, 341 
tensor form, 982 
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level lines, 905, 906 


particular: identity, null or zero, singular and 


Levi-Civita symbol, see €;;x, Levi-Civita symbol, non-singular, 249 


tensor 
likelihood function, 1255 
limits, 141-144 
definition, 141 
LH6pital’s rule, 142-144 
of functions containing exponents, 142 
of integrals, 59 
containing variables, 188 
of products, 141 
of quotients, 141-144 
of sums, 141 
line charge, electrostatic potential, 872, 878 
line integrals 
and Cauchy integrals, 851-853 
and Stokes’ theorem, 406-409 
of scalars, 377-387 
of vectors, 377-389 
physical examples, 381 
round closed loop, 386 
line of steepest descents, 908 
line, vector equation of, 226 
linear dependence and independence 
definition in a vector space, 242 
of basis vectors, 217 
relationship with rank, 267 
linear differential operator £, 511, 545, 554 
adjoint £7, 559 
eigenfunctions, see eigenfunctions 
eigenvalues, see eigenvalues, of linear 
differential operators 
for Sturm-Liouville equation, 564-568 
Hermitian, 555, 559-564 
self-adjoint, 559 
linear equations, differential 
first-order ODE, 474 
general ODE, 490-517 
ODE with constant coefficients, 492-503 
ODE with variable coefficients, 503-517 


properties, 249 
inear vector spaces, see vector spaces 
ines of steepest descent, 906 
Liouville’s theorem, 853 
Ln of a complex number, 99, 834 
n (natural logarithm) 

Maclaurin series for, 140 

of a complex number, 99, 834 
og-likelihood function, 1258 
ongitudinal vibrations in a rod, 677 
ottery (UK), and hypergeometric distribution, 

1174 

ower triangular matrices, 269 


Maclaurin series, 138 
standard expressions, 140 
Madelung constant, 149 
magnetic dipole, 220 
magnitude of a vector, 218 
in terms of scalar or dot product, 221 
mappings between groups, see groups, mappings 
between 
marginal distributions, 1198 
mass of non-uniform bodies, 193 
matrices, 241-307 
as a vector space, 252 
as arrays of numbers, 249 
as representation of a linear operator, 249 
column, 250 
elements, 249 
minors and cofactors, 259 
identity or unit, 254 
row, 250 
zero or null, 254 
matrices, algebra of, 250 
addition, 251 
change of basis, 283-285 
Cholesky separation, 313 


linear equations, simultaneous, see simultaneous diagonalisation, see diagonalisation of 


linear equations 
linear independence of functions, 491 
Wronskian test, 491, 532 
linear integral operator K, 805 
and Schmidt—Hilbert theory, 816-818 
Hermitian conjugate, 805 
inverse, 806 


matrices 
multiplication, 252-254 
and common eigenvalues, 278 
commutator, 309 
non-commutativity, 254 
multiplication by a scalar, 251 
normal modes, see normal modes 


linear interpolation for algebraic equations, 988 numerical methods, see numerical methods 


linear least squares, method of, 1272 
linear molecules 
normal modes of, 320-322 
symmetries of, 1077 
linear operators, 247-249 
associativity, 249 
distributivity over addition, 249 
eigenvalues and eigenvectors, 272 
in a particular basis, 248 
inverse, 249 
non-commutativity, 249 


for simultaneous linear equations 
similarity transformations, see similarity 
transformations 
simultaneous linear equations, see 
simultaneous linear equations 
subtraction, 251 
matrices, derived 
adjoint, 256-258 
complex conjugate, 256-258 
Hermitian conjugate, 256-258 
inverse, see inverse matrices 
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transpose, 250 
matrices, properties of 
anti- or skew-symmetric, 270 
anti-Hermitian, see anti-Hermitian matrices 
determinant, see determinants 
diagonal, 268 
eigenvalues, see eigenvalues 
eigenvectors, see eigenvectors 
Hermitian, see Hermitian matrices 
normal, see normal matrices 
nullity, 293 
order, 249 
orthogonal, 270 
rank, 267 
square, 249 
symmetric, 270 
trace or spur, 258 
triangular, 269 
tridiagonal, 998-1000, 1030 
unitary, see unitary matrices 
matrix elements in quantum mechanics 
as integrals, 1103 
dipole, 1108, 1115 
maxima and minima (local) of a function of 
constrained variables, see Lagrange 
undetermined multipliers 
one real variable, 50-52 
sufficient conditions, 51 
several real variables, 162-167 
sufficient conditions, 164, 167 
maximum modulus theorem, 881 
maximum-likelihood, method of, 1255-1271 
and Bayesian approach, 1264 
bias, 1260 
data modelling, 1255 
estimator, 1256 
extended, 1270 
og-likelihood function, 1258 
parameter estimation, 1255 
ransformation invariance, 1260 
Maxwell’s 
electromagnetic equations, 373, 408, 979 
hermodynamic relations, 176-178 
Maxwell—Boltzmann statistics, 1138 
mean [Lt 
rom MGF, 1163 
rom PGF, 1158 
of RVD, 1144 
of sample, 1223 
of sample: geometric, harmonic, root mean 
square, 1223 
mean value of a function of 
one variable, 72 
several variables, 199 
mean value theorem, 56 
median of RVD, 1145 
membrane 
deformed rim, 725-727 
normal modes, 739, 1112 
transverse vibrations, 677, 739, 768, 1112 


method of images, 706, 758-765, 878 
disc (section of cylinder), 764, 766 
infinite plate, 759 
intersecting plates in two dimensions, 761 
sphere, 762-764, 772 

metric tensor, 957-960, 963 
and Christoffel symbols, 966 
and scale factors, 957, 972 
covariant derivative of, 982 
determinant, 957, 964 

derivative of, 973 
length element, 957 
raising or lowering index, 959, 963 
scalar product, 958 
volume element, 957, 981 

MGF, see moment generating functions 

Milne’s method, 1022 

minimum-variance estimator, 1232 

minor of a matrix element, 259 

mixed, components of tensor, 957, 962, 969 

ML estimators, 1256 
bias, 1260 
confidence limits, 1262 
efficiency, 1261 
transformation invariance, 1260 

mode of RVD, 1145 

modulo, mod N, multiplication, 1049 

modulus 
of a complex number, 87 
of a vector, see magnitude of a vector 

molecules 
bonding in, 1103, 1105-1108 
dipole moments of, 1077 
symmetries of, 1077 
moment generating functions (MGFs), 
1162-1167 
and central limit theorem, 1195 
and PGF, 1163 
mean and variance, 1163 
particular distributions 
binomial, 1170 
exponential, 1191 
Gaussian, 1163, 1185 
Poisson, 1177 
properties, 1163 

moments (of distributions) 
central, 1148 
of RVD, 1147 

moments (of forces), vector representation of, 

223 

moments of inertia 
and inertia tensor, 951 
definition, 198 
of disc, 208 
of rectangular lamina, 198 
of right circular cylinder, 209 
of sphere, 205 
perpendicular axes theorem, 209 

momentum as first-order tensor, 933 

monomorphism, 1061 
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Monte Carlo methods, of integration, 1009-1017 


antithetic variates, 1014 
control variates, 1013 
crude, 1011 
hit or miss, 1014 
importance sampling, 1012 
multiple integrals, 1016 
random number generation, 1017 
stratified sampling, 1012 
Morera’s theorem, 851 
multinomial distribution, 1208 
and multiple Poisson distribution, 1218 
multiple angles, trigonometric formulae, 10 
multiple integrals 
application in finding 
area and volume, 191-193 
mass, centre of mass and centroid, 193-195 
mean value of a function of several 
variables, 199 
moments of inertia, 198 
change of variables 
double integrals, 200-204 
general properties, 206 
triple integrals, 204 
definitions of 
double integrals, 187 
triple integrals, 190 
evaluation, 188-190 
notation, 188, 189, 191 
order of integration, 188, 191 
multiplication tables for groups, see group 
multiplication tables 
multiplication theorem, see Parseval’s theorem 
multivalued functions, 835-837 
integration of, 865-867 
multivariate distributions, 1196, 1207-1211 
change of variables, 1206 
Gaussian, 1209 
multinomial, 1208 
mutually exclusive events, 1120, 1129 


n(x), see spherical Bessel functions 
nabla V, see gradient operator (grad) 
natural interval 
for associated Laguerre equation, 567, 622 
for associated Legendre equation, 567, 590, 
591 
for Bessel equation, 608 
for Chebyshev equation, 567, 599 
for Hermite equation, 567 
for Laguerre equation, 567, 619 
for Legendre equation, 567, 583 
for simple harmonic oscillator equation, 567 
for Sturm—Liouville equations, 565, 567 
natural logarithm, see In and Ln 
natural numbers, in series, 31, 121 
natural representations, 1081, 1110 
necessary and sufficient conditions, 34 
negative binomial distribution, 1172 
negative function, 556 


negative vector, 242 
Neumann boundary conditions, 702 
Green’s functions, 754, 765-767 
method of images, 765-767 
self-consistency, 765 
Neumann functions Y,(x), 607 
Neumann series, 813-815 
Newton—Raphson (NR) method, 990-992 
order of convergence, 993 
Neyman-—Pearson test, 1280 
nodes of oscillation, 693 
non-Abelian groups, 1052-1056 
of functions, 1055 
of matrices, 1054 
of permutations, 1056-1058 
of rotations and reflections, 1052 
non-Cartesian coordinates, see curvilinear, 
cylindrical polar, plane polar and spherical 
polar coordinates 
non-linear differential equations, see ordinary 
differential equations, non-linear 
non-linear least squares, method of, 1276 
norm of 
function, 557 
vector, 244 
norma 
to coordinate surface, 366 
to plane, 228 
to surface, 346, 350, 390 
normal derivative, 350 
normal distribution, see Gaussian (normal) 
distribution 
normal matrices, 272 
eigenvectors 
completeness, 275 
orthogonality, 275 
eigenvectors and eigenvalues, 273-276 
normal modes, 316-329 
characteristic equation, 319 
coupled pendulums, 329, 331 
definition, 320 
degeneracy, 1110-1113 
frequencies of, 319 
linear molecular system, 320-322 
membrane, 739, 1112 
normal coordinates, 320 
normal equations, 320 
rod-string system, 317-320 
symmetries of, 322 
normal subgroups, 1063 
normalisation of 
eigenfunctions, 562 
eigenvectors, 273 
functions, 557 
vectors, 219 
null (zero) 
matrix, 254, 255 
operator, 249 
space, of a matrix, 293 
vector, 214, 242, 556 
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null operation, as identity element of group, 
1044 
nullity, of a matrix, 293 
numerical methods for algebraic equations, 
985-992 
binary chopping, 990 
convergence of iteration schemes, 992-994 
linear interpolation, 988 
Newton-Raphson, 990-992 
rearrangement methods, 987 
numerical methods for integration, 1000-1009 
Gaussian integration, 1005-1009 
mid-point rule, 1034 
Monte Carlo, 1009 
nomenclature, 1001 
Simpson’s rule, 1004 
trapezium rule, 1002-1004 
numerical methods for ordinary differential 
equations, 1020-1030 
accuracy and convergence, 1021 
Adams method, 1024 
difference schemes, 1021-1023 
Euler method, 1021 
first-order equations, 1021-1028 
higher-order equations, 1028-1030 
isoclines, 1028 
Milne’s method, 1022 
prediction and correction, 1024-1026 
reduction to matrix form, 1030 
Runge-Kutta methods, 1026-1028 
Taylor series methods, 1023 
numerical methods for partial differential 
equations, 1030-1032 
diffusion equation, 1032 
Laplace’s equation, 1031 
minimising error, 1032 
numerical methods for simultaneous linear 
equations, 994-1000 
Gauss-Seidel iteration, 996-998 
Gaussian elimination with interchange, 995 
matrix form, 994-1000 
tridiagonal matrices, 998-1000 


O(x), order of, 132 
observables in quantum mechanics, 277, 560 
odd functions, see antisymmetric functions 


ODE, see ordinary differential equations (ODEs) 


operators 
Hermitian, see Hermitian operators 
linear, see linear operators and linear 
differential operator and linear integral 
operator 
operators (quantum) 
angular momentum, 656-663 
annihilation and creation, 667 
coordinate-free, 648-671 
eigenvalues and eigenstates, 649 
physical examples 
angular momentum, 658 
Hamiltonian, 657 


order of 
approximation in Taylor series, 137n 
convergence of iteration schemes, 993 
group, 1043 
group element, 1047 
ODE, 468 
permutation, 1058 
recurrence relations (series), 497 
subgroup, 1061 
and Lagrange’s theorem, 1065 
tensor, 930 
ordinary differential equations (ODE), see also 
differential equations, particular 
boundary conditions, 468, 470, 501 
complementary function, 491 
degree, 468 
dimensionally homogeneous, 475 
exact, 472, 505 
first-order, 468-484 
first-order higher-degree, 480-484 
soluble for p, 480 
soluble for x, 481 
soluble for y, 482 
general form of solution, 468-470 
higher-order, 490-523 
homogeneous, 490 
inexact, 473 
isobaric, 476, 521 
linear, 474, 490-517 
non-linear, 518-523 
exact, 519 
isobaric (homogeneous), 521 
x absent, 518 
y absent, 518 
order, 468 
ordinary point, see ordinary points of ODE 
particular integral (solution), 469, 492, 494 
singular point, see singular points of ODE 
singular solution, 469, 481, 482, 484 
ordinary differential equations, methods for 
canonical form for second-order equations, 
516 
eigenfunctions, 554-573 
equations containing linear forms, 478-480 
equations with constant coefficients, 492-503 
Green’s functions, 511-516 
integrating factors, 473-475 
Laplace transforms, 501-503 
numerical, 1020-1030 
partially known CF, 506 
separable variables, 471 
series solutions, 531-550, 604 
undetermined coefficients, 494 
variation of parameters, 508-510 
ordinary points of ODE, 533, 535-538 
indicial equation, 543 
orthogonal lines, condition for, 12 
orthogonal matrices, 270, 929, 930 
general properties, see unitary matrices 
orthogonal systems of coordinates, 364 
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orthogonal transformations, 932 
orthogonalisation (Gram—Schmidt) of 


eigenfunctions of an Hermitian operator, 562 


eigenvectors of a normal matrix, 275 
functions in a Hilbert space, 557 
orthogonality of 
eigenfunctions of an Hermitian operator, 
561-563 
eigenvectors of a normal matrix, 275 
eigenvectors of an Hermitian matrix, 277 
functions, 557 
terms in Fourier series, 417, 425 
vectors, 219, 244 
orthogonality properties of characters, 1094, 
1102 
orthogonality theorem for irreps, 1090-1092 
orthonormal 
basis functions, 557 
basis vectors, 244 
under unitary transformation, 285 
oscillations, see normal modes 
outcome, of trial, 1119 
outer product of two vectors, 936 


P;(x), see Legendre polynomials 
P/"(x), see associated Legendre functions 
Pappus’ theorems, 195-197 
parabola, equation for, 16 
parabolic PDE, 687, 690 
parallel axis theorem, 238 
parallel vectors, 223 
parallelepiped, volume of, 225 
parallelogram equality, 247 
parallelogram, area of, 223, 224 
parameter estimation (statistics), 1229-1255, 
1298 
Bessel correction, 1248 
error in mean, 1298 
maximum-likelihood, 1255 
mean, 1243 
variance, 1245-1248 
parameters, variation of, 508-510 
parametric equations 
of conic sections, 17 
of cycloid, 370, 785 
of space curves, 340 
of surfaces, 345 
parity inversion, 1102 
Parseval’s theorem 
conservation of energy, 451 
for Fourier series, 426 
for Fourier transforms, 450 
partial derivative, see partial differentiation 
partial differential equations (PDE), 675-707, 
713-767, see also differential equations, 
particular 
arbitrary functions, 680-685 
boundary conditions, 681, 699-707, 723 
characteristics, 699-705 
and equation type, 703 


equation types, 687, 710 
first-order, 681-687 
general solution, 681-692 
homogeneous, 685 
inhomogeneous equation and problem, 
685-687, 744-746, 751-767 
particular solutions (integrals), 685-692 
second-order, 687-698 
partial differential equations (PDE), methods for 
change of variables, 691, 696-698 
constant coefficients, 687 
general solution, 689 
integral transform methods, 747-751 
method of images, see method of images 
numerical, 1030-1032 
separation of variables, see separation of 
variables 
superposition methods, 717-724 
with no undifferentiated term, 684 
partial differentiation, 151-179 
as gradient of a function of several real 
variables, 151 
chain rule, 157 
change of variables, 158-160 
definitions, 151-153 
properties, 157 
cyclic relation, 157 
reciprocity relation, 157 
partial fractions, 18-25 
and degree of numerator, 21 
as a means of integration, 64 
complex roots, 22 
in inverse Laplace transforms, 454, 502 
repeated roots, 23 
partial sum, 115 
particular integrals (PI), 469, see also ordinary 
differential equation, methods for and 
partial differential equations, methods for 
partition of a 
group, 1064 
set, 1065 
parts, integration by, 67-69 
path integrals, see line integrals 
PDE, see partial differential equations 
PDFs, 1140 
pendulums, coupled, 329, 331 
periodic function representation, see Fourier 
series 
permutation groups Sy, 1056-1058 
cycle notation, 1057 
permutation law in a group, 1047 
permutations, 1133-1139 
degree, 1056 
distinguishable, 1135 
order of, 1058 
symbol "P;, 1133 
perpendicular axes theorem, 209 
perpendicular vectors, 219, 244 
PF, see probability functions 
PGF, see probability generating functions 
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phase memory, 895 
phase, complex, 896 
PI, see particular integrals 
plane curves, length of, 73 
in Cartesian coordinates, 73 
in plane polar coordinates, 74 
plane polar coordinates, 70, 336 
arc length, 74, 361 
area element, 202, 361 
basis vectors, 336 
velocity and acceleration, 337 
plane waves, 695, 716 
planes 
and simultaneous linear equations, 300 
vector equation of, 227 
plates, conducting, see also complex potentials, 
for plates 
line charge near, 761 
point charge near, 759 
point charges, 6-function respresentation, 441 
point groups, 1082 
points of inflection of a function of 
one real variable, 50-52 
several real variables, 162-167 
Poisson distribution Po(/), 1174-1179 
and Gaussian distribution, 1187 
as limit of binomial distribution, 1174, 1177 
mean and variance, 1176 
MGF, 1177 
multiple, 1178 
recurrence formula, 1176 
Poisson equation, 575, 679, 744-746 
fundamental solution, 757 
Green’s functions, 753-767 
uniqueness, 705-707 
Poisson summation formula, 461 
Poisson’s ratio, 953 
polar coordinates, see plane polar and cylindrical 
polar and spherical polar coordinates 
polar representation of complex numbers, 92-95 
polar vectors, 949 
pole, of a function of a complex variable 
contours containing, 861-867, 884-887 
order, 837, 856 
residue, 856-858 
polynomial equations, 1-10 
conjugate roots, 99 
actorisation, 7 
multiplicities of roots, 4 
number of roots, 83, 85, 868 
properties of roots, 9 
real roots, 1 
solution of, using de Moivre’s theorem, 98 
polynomial solutions of ODE, 538, 548-550 
populations, sampling of, 1222 
positive (semi-) definite quadratic and Hermitian 
forms, 290 
positive semi-definite norm, 244 
potential energy of 
ion in a crystal lattice, 148 


magnetic dipole in a field, 220 
oscillating system, 317 
potential function 
and conservative fields, 389 
complex, 871-876 
electrostatic, see electrostatic fields and 
potentials 
gravitational, see gravitational fields and 
potentials 
vector, 389 
power series 
and differential equations, see series solutions 
of differential equations 
interval of convergence, 132 
Maclaurin, see Maclaurin series 
manipulation: difference, differentiation, 
integration, product, substitution, sum, 134 
Taylor, see Taylor series 
power series in a complex variable, 133, 830-832 
analyticity, 832 
circle and radius of convergence, 133, 831 
convergence tests, 831, 832 
form, 830 
power, in hypothesis testing, 1280 
powers, complex, 99, 833 
prediction and correction methods, 1024-1026, 
1035 
prime, non-existence of largest, 34 
principal axes of 
Cartesian tensors, 951-953 
conductivity tensors, 952 
inertia tensors, 951 
quadratic surfaces, 292 
rotation symmetry, 1102 
principal normals of space curves, 342 
principal value of 
complex integrals, 864 
complex logarithms, 100, 834 
principle of the argument, 880 
probability, 1124-1211 
axioms, 1125 
conditional, 1128-1133 
Bayes’ theorem, 1132 
combining, 1130 
definition, 1125 
for intersection MN, 1120 
for union U, 1121, 1125-1128 
probability distributions, 1139, see also 
individual distributions 
bivariate, see bivariate distributions 
change of variables, 1150-1157 
cumulants, 1166 
generating functions, see moment generating 
functions and probability generating 
functions 
mean p, 1144 
mean of functions, 1145 
mode, median and quartiles, 1145 
moments, 1147-1150 
multivariate, see multivariate distributions 
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standard deviation o, 1146 
variance a7, 1146 
probability functions (PFs), 1139 
cumulative (CPFs), 1139, 1141 
density functions (PDFs), 1140 
probability generating functions (PGFs), 
1157-1162 
and MGF, 1163 
binomial, 1161 
definition, 1158 
geometric, 1159 
mean and variance, 1158 
Poisson, 1158 
sums of RV, 1161 
trials, 1158 
variable sums of RV, 1161 
product rule for differentiation, 44-46, 48-50 
products of inertia, 951 
projection operators for irreps, 1107, 1116 
projection tensors, 979 
proper rotations, 946 
proper subgroups, 1061 
pseudoscalars, 947, 950 
pseudotensors, 946-950, 964 
pseudovectors, 946-950 


Q/(x), see Legendre polynomials 
Q'(x), see associated Legendre functions 
quadratic equations 
complex roots of, 83 
properties of roots, 10 
roots of, 2 
quadratic forms, 288-292 
completing the square, 1206 
positive definite and semi-definite, 290 
quadratic surfaces, 292 
removing cross terms, 289 
stationary properties of eigenvectors, 290 
quantum mechanics, from classical mechanics, 
657 
quantum operators, see operators (quantum) 
quartiles, of RVD, 1145 
quaternion group, 1073, 1113 
quotient law for tensors, 939-941 
quotient rule for differentiation, 47 
quotient test for series, 127 


radius of convergence, 133, 831 
radius of curvature 
of plane curves, 53 
of space curves, 342 
radius of torsion of space curves, 343 
random number generation, 1017 
random numbers, non-uniform distribution, 
1035 
random variable distributions, see probability 
distributions 
random variables (RV), 1119, 1139-1143 
continuous, 1140-1143 
dependent, 1196-1205 


discrete, 1139 
independent, 1156, 1200 
sums of, 1160-1162 
uncorrelated, 1200 
range of a matrix, 293 
rank of matrices, 267 
and determinants, 267 
and linear dependence, 267 
rank of tensors, see order of tensor 
rate of change of a function of 
one real variable, 41 
several real variables, 153-155 
ratio comparison test, 127 
ratio test (D’Alembert), 126, 832 
in convergence of power series, 132 
ratio theorem, 215 
and centroid of a triangle, 216 
Rayleigh—Ritz method, 327-329, 800 
real part or term of a complex number, 83 
real roots, of a polynomial equation, 1 
rearrangement methods for algebraic equations, 
987 
reciprocal vectors, 233, 366, 955, 959 
reciprocity relation for partial derivatives, 157 
rectangular distribution, 1194 
Fourier transform of, 442 
recurrence relations (functions), 585, 611 
associated Laguerre polynomials, 624 
associated Legendre functions, 592 
Chebyshev polynomials, 601 
confluent hypergeometric functions, 635 
Hermite polynomials, 628 
hypergeometric functions, 632 
Laguerre polynomials, 620 
Legendre polynomials, 586 
recurrence relations (series), 496-501 
characteristic equation, 499 
coefficients, 536, 538, 999 
first-order, 497 
second-order, 499 
higher-order, 501 
reducible representations, 1084, 1086 
reduction formulae for integrals, 69 
reflections 
and improper rotations, 946 
as symmetry operations, 1041 
reflexivity, and equivalence relations, 1064 
regular functions, see analytic functions 
regular representations, 1097, 1110 
regular singular points, 534, 538-540 
relative velocities, 218 
remainder term in Taylor series, 138 
repeated roots of auxiliary equation, 493 
representation, 1076 
definition, 1082 
dimension of, 1078, 1082 
equivalent, 1084-1086 
faithful, 1083, 1098 
generation of, 1078-1084, 1112 
irreducible, see irreps 
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natural, 1081, 1110 
product, 1103-1105 
reducible, 1084, 1086 
regular, 1097, 1110 
counting irreps, 1098 
unitary, 1086 
representative matrices, 1079 
block-diagonal, 1086 
eigenvalues, 1100 
inverse, 1083 
number needed, and order of group, 1082 
of identity, 1082 
residue 
at a pole, 856-858 
theorem, 858-860 
resolution function, 446 
resolvent kernel, 814, 815 
response matrix, for linear least squares, 1273 
rhomboid, volume of, 237 
Riemann tensor, 981 
Riemann theorem for conditional convergence, 
124 
Riemann zeta series, 128, 129 
right hand screw rule, 222 
Rodrigues’ formula for 
associated Laguerre polynomials, 622 
associated Legendre functions, 588 
Chebyshev polynomials, 599 
Hermite polynomials, 626 
Laguerre polynomials, 618 
Legendre polynomials, 581 
Rolle’s theorem, 55 
root test (Cauchy), 129, 831 
roots 
of a polynomial equation, 2 
properties, 9 
of unity, 97 
rope, suspended at its ends, 786 
rotation groups (continuous), invariant 
subspaces, 1088 
rotation matrices as a group, 1048 
rotation of a vector, see curl 
rotations 
as symmetry operations, 1041 
axes and orthogonal matrices, 930, 931, 961 
improper, 946-948 
invariance under, 934 
product of, 931 
proper, 946 
Rouché’s theorem, 880-882 
row matrix, 250 
Runge-Kutta methods, 1026-1028 
RV, see random variables 
RVD (random variable distributions), see 
probability distributions 


saddle point method of integration, 908 
saddle points, 162 
and integral evaluation, 905 
sufficient conditions, 164, 167 


sampling 


correlation, 1227 

covariance, 1227 

space, 1119 

statistics, 1222-1229 

with or without replacement, 1129 


scalar fields, 347 


derivative along a space curve, 349 
gradient, 348-352 

line integrals, 377-387 

rate of change, 349 


scalar product, 219-222 


and inner product, 244 

and metric tensor, 958 

and perpendicular vectors, 219, 244 

for vectors with complex components, 221 
in Cartesian coordinates, 221 

invariance, 930, 939 


scalar triple product, 224-226 


cyclic permutation of, 225 

in Cartesian coordinates, 225 
determinant form, 225 

interchange of dot and cross, 225 


scalars, 212 


invariance, 930 
zero-order tensors, 933 


scale factors, 359, 362, 364 


and metric tensor, 957, 972 


scattering in quantum mechanics, 463 
Schmidt—Hilbert theory, 816-819 
Schrédinger equation, 679 


constant potential, 768 
hydrogen atom, 741 
numerical solution, 1039 
variational approach, 795 


Schwarz inequality, 246, 559 
Schwarz-—Christoffel transformation, 843 

second differences, 1019 

second-order differential equations, see ordinary 


differential equations and partial differential 
equations 


secular determinant, 280 
self-adjoint operators, 559-564, see also 


Hermitian operators 


semicircle, angle in, 18 

semicircular lamina, centre of mass, 197 
separable kernel in integral equations, 807 
separable variables in ODE, 471 
separation constants, 715, 717 

separation of variables, for PDE, 713-746 


1328 


diffusion equation, 716, 722-724, 737, 751 
expansion methods, 741-744 

general method, 713-717 

Helmholtz equation, 737-741 

inhomogeneous boundary conditions, 722-724 
inhomogeneous equations, 744-746 

Laplace equation, 717-722, 725-737, 741 
polar coordinates, 725-746 

separation constants, 715, 717 

superposition methods, 717-724 
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wave equation, 714-716, 737, 739 
series, 115-141 
convergence of, see convergence of infinite 
series 
differentiation of, 131 
finite and infinite, 116 
integration of, 131 
multiplication by a scalar, 131 
multiplication of (Cauchy product), 131 
notation, 116 
operations, 131 
summation, see summation of series 
series, particular 
arithmetic, 117 
arithmetico-geometric, 118 
Fourier, see Fourier series 
geometric, 117 
Maclaurin, 138, 140 
power, see power series 
powers of natural numbers, 121 
Riemann zeta, 128, 129 
Taylor, see Taylor series 
series solutions of differential equations, 531-550 
about ordinary points, 535—538 
about regular singular points, 538-540 
Frobenius series, 539 
convergence, 536 
indicial equation, 540 
linear independence, 540 
polynomial solutions, 538, 548-550 
recurrence relation, 536, 538 
second solution, 537, 544-548 
derivative method, 545-548 
Wronskian method, 544, 580 
shortest path, 778 
and geodesics, 976, 982 
similarity transformations, 283-285, 929, 1092 
properties of matrix under, 284 
unitary transformations, 285 
simple harmonic oscillator, 555 
energy levels of, 642, 902 
equation, 535, 566 
operator formalism, 667 
simple poles, 838 
Simpson’s rule, 1004 
simultaneous linear equations, 292-307 
and intersection of planes, 300 
homogeneous and inhomogeneous, 293 
singular value decomposition, 301—307 
solution using 
Cramer’s rule, 299 
inverse matrix, 295 
numerical methods, see numerical methods 
for simultaneous linear equations 
sine, sin(x) 
in terms of exponential functions, 102 
Maclaurin series for, 140 
orthogonality relations, 417 
singular and non-singular 
integral equations, 805 


linear operators, 249 
matrices, 263 
singular integrals, see improper integrals 
singular point (singularity), 826, 837-839 
essential, 838, 856 
removable, 838 
singular points of ODE, 533 
irregular, 534 
particular equations, 535 
regular, 534 
singular solution of ODE, 469, 481, 482, 484 
singular value decomposition 
and simultaneous linear equations, 301-307 
singular values, 302 
sinh, hyperbolic sine, 102, 833, see also 
hyperbolic functions 
skew-symmetric matrices, 270 
skewness, 1150, 1227 
Snell’s law, 788 
soap films, 780 
solenoidal vectors, 352, 389 
solid angle 
as surface integral, 395 
subtended by rectangle, 411 
solid: mass, centre of mass and centroid, 
193-195 
source density, 679 
space curves, 340-344 
arc length, 341 
binormal, 342 
curvature, 342 
Frenet—Serret formulae, 343 
parametric equations, 340 
principal normal, 342 
radius of curvature, 342 
radius of torsion, 343 
tangent vector, 342 
torsion, 342 
spaces, see vector spaces 
span of a set of vectors, 242 
sphere, vector equation of, 228 
spherical Bessel functions, 615, 741 
of first kind j,(z), 615 
of second kind n;(z), 615 
spherical harmonics Y/"(0, b), 593-595 
addition theorem, 594 
spherical polar coordinates, 361-363 
area element, 362 
basis vectors, 362 
length element, 362 
vector operators, 361-363 
volume element, 205, 362 
spur of a matrix, see trace of a matrix 
square matrices, 249 
square, symmetries of, 1100 
square-wave, Fourier series for, 418 
stagnation points of fluid flow, 873 
standard deviation o, 1146 
of sample, 1224 
standing waves, 693 


1329 


INDEX 


stationary phase, method of, 912-920 
stationary values 
of functions of 
one real variable, 50-52 
several real variables, 162-167 
of integrals, 776 


under constraints, see Lagrange undetermined 


multipliers 
statistical tests, and hypothesis testing , 1278 
statistics, 1119, 1221-1298 
describing data, 1222-1229 
estimating parameters, 1229-1255, 1298 
steepest descents, method of, 908-912 
Stirling’s 
approximation, 637, 1185 
asymptotic series, 637 
okes constant, in Stokes phenomenon, 904 
okes line, 899, 903 
okes phenomenon, 903 
dominant term, 904 
Stokes constant, 904 
subdominant term, 904 
okes’ equation, 643, 799, 888-894 
Airy integrals, 890-894 
qualitative solutions, 888 
series solution, 890 
okes’ theorem, 388, 406-409 
for tensors, 955 
physical applications, 408 
related theorems, 407 
strain tensor, 953 
stratified sampling, in Monte Carlo methods, 
1012 
streamlines and complex potentials, 873 
stress tensor, 953 
stress waves, 980 
string 
loaded, 798 
plucked, 770 
transverse vibrations of, 676, 789 
Student’s t-distribution 
normalisation, 1286 
plots, 1287 
Student’s t-test, 1284-1290 
comparison of means, 1289 
critical points table, 1288 
one- and two-tailed confidence limits, 1288 
Sturm-Liouville equations, 564 
boundary conditions, 564 
examples, 566 
associated Laguerre, 566, 622 
associated Legendre, 566, 590, 591 
Bessel, 566, 608-611 
Chebyshev, 566, 599 
confluent hypergeometric, 566 
Hermite, 566 
hypergeometric, 566 
Laguerre, 566, 619 
Legendre, 566, 583 
manipulation to self-adjoint form, 565-568 


Ann 


n 


n 


natural interval, 565, 567 

two independent variables, 801 
variational approach, 790-795 
weight function, 790 

zeros of eigenfunctions, 573 


subdominant term, in Stokes phenomenon, 904 


subgroups, 1061-1063 
index, 1066 
normal, 1063 
order, 1061 
Lagrange’s theorem, 1065 
proper, 1061 
trivial, 1061 
submatrices, 267 
subscripts and superscripts, 928 
contra- and covariant, 956 
covariant derivative, 969 
dummy, 928 
free, 928 
partial derivative, 969 
summation convention, 928, 955 
substitution, integration by, 65-67 
summation convention, 928, 955 
summation of series, 116-124 
arithmetic, 117 
arithmetico-geometric, 118 
contour integration method, 882 
difference method, 119 
Fourier series method, 427 
geometric, 117 
powers of natural numbers, 121 
transformation methods, 122-124 
differentiation, 122 
integration, 122 
substitution, 123 
superposition methods 
for ODE, 554, 568-571 
for PDE, 717-724 
surface integrals 
and divergence theorem, 401 
Archimedean upthrust, 396, 410 
of scalars, vectors, 389-396 
physical examples, 395 
surfaces, 345-347 
area of, 346 
cone, 74 
solid, and Pappus’ theorem, 195-197 
sphere, 346 
coordinate curves, 346 
normal to, 346, 350 
of revolution, 74 
parametric equations, 345 
quadratic, 292 
tangent plane, 346 
SVD, see singular value decomposition 
symmetric functions, 416 
and Fourier series, 419 
and Fourier transforms, 445 
symmetric matrices, 270 
general properties, see Hermitian matrices 
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symmetric tensors, 938 
symmetry in equivalence relations, 1064 
symmetry operations 

on molecules, 1041 

order of application, 1044 


t-test, see Student’s t-test 
t substitution, 65 
tan—! x, Maclaurin series for, 140 
tangent planes to surfaces, 346 
tangent vectors to space curves, 342 
tanh, hyperbolic tangent, see hyperbolic 
functions 
Taylor series, 136-141 
and finite differences, 1019, 1026 
and Taylor’s theorem, 136-139, 853 
approximation errors, 139 
in numerical methods, 992, 1003 
as solution of ODE, 1023 
for functions of a complex variable, 853-855 


for functions of several real variables, 160-162 


remainder term, 138 
required properties, 136 
standard forms, 136 
tensors, see Cartesian tensors and Cartesian 
tensors, particular and general tensors 
test statistic, 1278 
tetrahedral group, 1115 
tetrahedron 
mass of, 194 
volume of, 192 
thermodynamics 
first law of, 176 
Maxwell’s relations, 176-178 
top-hat function, see rectangular distribution 
torque, vector representation of, 223 
torsion of space curves, 342 
total derivative, 154 
total differential, 154 
trace of a matrix, 258 
and second-order tensors, 939 
as sum of eigenvalues, 280, 287 
invariance under similarity transformations, 
284, 1092 
trace formula, 287 
transcendental equations, 986 
transformation matrix, 283, 289 
transformations 
active and passive, 948 
conformal, 839-844 
coordinate, see coordinate transformations 
similarity, see similarity transformations 
transforms, integral, see integral transforms and 
Fourier transforms and Laplace transforms 
transients, and the diffusion equation, 723 
transitivity in equivalence relations, 1064 
transpose of a matrix, 250, 255 
product rule, 256 
transverse vibrations 
membrane, 677, 739, 768 


rod, 769 
string, 676 
trapezium rule, 1002-1004 
trial functions 
for eigenvalue estimation, 793 
for particular integrals of ODE, 494 
trials, 1119 
triangle inequality, 246, 559 
triangle, centroid of, 216 
triangular matrices, 269 
tridiagonal matrices, 998-1000, 1030, 1033 
trigonometric identities, 10-15 
triple integrals, see multiple integrals 
triple scalar product, see scalar triple product 
triple vector product, see vector triple product 
turning point, 50 


uncertainty principle (Heisenberg), 435-437 
from commutator, 664 
undetermined coefficients, method of, 494 
undetermined multipliers, see Lagrange 
undetermined multipliers 
uniform distribution, 1194 
union U, probability for, 1121, 1125, 1128 
uniqueness theorem 
Klein—Gordon equation, 711 
Laplace equation, 741 
Poisson equation, 705-707 
unit step function, see Heaviside function 
unit vectors, 219 
unitary 
matrices, 271 
eigenvalues and eigenvectors, 278 
representations, 1086 
transformations, 285 
upper triangular matrices, 269 


variable end-points, 782-785 
variable, dummy, 61 
variables, separation of, see separation of 
variables 
variance o”, 1146 
from MGF, 1163 
from PGF, 1159 
of dependent RV, 1203 
of sample, 1224 
variation of parameters, 508-510 
variation, constrained, 785-787 
variational principles, physical, 787-790 
Fermat, 787 
Hamilton, 788 
variations, calculus of, see calculus of variations 
vector operators, 347-369 
acting on sums and products, 354 
combinations of, 355-357 
curl, 353, 368 
del V, 348 
del squared V2, 352 
divergence (div), 352 
geometrical definitions, 398-400 
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gradient operator (grad), 348-352, 367 
identities, 356, 978 
Laplacian, 352, 368 
non-Cartesian, 357-369 
tensor forms, 971-975 
curl, 974 
divergence, 972 
gradient, 972 
Laplacian, 973 
vector product, 222-224 
anticommutativity, 222 
definition, 222 
determinant form, 224 
in Cartesian coordinates, 224 
non-associativity, 222 
vector spaces, 242-247, 1113 
associativity of addition, 242 
basis vectors, 243 
commutativity of addition, 242 
complex, 242 
defining properties, 242 
dimensionality, 243 
group actions on, 1088 
inequalities: Bessel, Schwarz, triangle, 246 
invariant, 1088, 1113 
matrices as an example, 252 
of infinite dimensionality, 556-559 
associativity of addition, 556 
basis functions, 556 
commutativity of addition, 556 
defining properties, 556 
Hilbert spaces, 557-559 
inequalities: Bessel, Schwarz, triangle, 559 
parallelogram equality, 247 
real, 242 
span of a set of vectors in, 242 
vector triple product, 226 
identities, 226 
non-associativity, 226 
vectors 
as first-order tensors, 932 
as geometrical objects, 241 
base, 336 
column, 250 
compared with scalars, 212 
component form, 217 
examples of, 212 
graphical representation of, 212 
irrotational, 353 
magnitude of, 218 
non-Cartesian, 336, 358, 362 
notation, 212 
polar and axial, 949 
solenoidal, 352, 389 
span of, 242 
vectors, algebra of, 212-234 
addition and subtraction, 213 
in component form, 218 
angle between, 221 


associativity of addition and subtraction, 213 


commutativity of addition and subtraction, 
213 
multiplication by a complex scalar, 222 
multiplication by a scalar, 214 
multiplication of, see scalar product and 
vector product 
outer product, 936 
vectors, applications 
centroid of a triangle, 216 
equation of a line, 226 
equation of a plane, 227 
equation of a sphere, 228 
finding distance from a 
line to a line, 231 
line to a plane, 232 
point to a line, 229 
point to a plane, 230 
intersection of two planes, 228 
vectors, calculus of, 334-369 
differentiation, 334-339, 344 
integration, 339 
line integrals, 377-389 
surface integrals, 389-396 
volume integrals, 396 
vectors, derived quantities 
curl, 353 
derivative, 334 
differential, 338, 344 
divergence (div), 352 
reciprocal, 233, 366, 955, 959 
vector fields, 347 
curl, 406 
divergence, 352 
flux, 395 
rate of change, 350 
vectors, physical 
acceleration, 335 
angular momentum, 238 
angular velocity, 223, 238, 353 
area, 393-395, 408 
area of parallelogram, 223, 224 
force, 212, 213, 220 
moment or torque of a force, 223 
velocity, 335 
velocity vectors, 335 
Venn diagrams, 1119-1124 
vibrations 
internal, see normal modes 
longitudinal, in a rod, 677 
transverse 
membrane, 677, 739, 768, 799, 801 
rod, 769 
string, 676, 789 
Volterra integral equation, 804, 805 
differentiation methods, 812 
Laplace transform methods, 810 
volume elements 
curvilinear coordinates, 365 
cylindrical polars, 360 
spherical polars, 205, 362 
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volume integrals, 396 zero-order tensors, 932-935 
and divergence theorem, 401 zeros of a function of a complex variable, 839 
volume of location of, 879-882, 921 
cone, 75 order, 839, 856 
ellipsoid, 207 principle of the argument, 880 
parallelepiped, 225 Rouché’s theorem, 880, 882 
rhomboid, 237 zeros of Sturm-Liouville eigenfunctions, 573 
tetrahedron, 192 zeros, of a polynomial, 2 
volumes zeta series (Riemann), 128, 129 
as surface integrals, 397, 401 z-plane, see Argand diagram 


in many dimensions, 210 

of regions, using multiple integrals, 191-193 
volumes of revolution, 75 

and surface area & centroid, 195-197 


wave equation, 676, 688, 790 
boundary conditions, 693-695 
characteristics, 704 
from Maxwell’s equations, 373 
in one dimension, 689, 693-695 
in three dimensions, 695, 714, 737 
standing waves, 693 

wave number, 437, 693n 

wave packet, 436 

wave vector, k, 437 

wavefunction of electron in hydrogen atom, 208 

Weber functions Y,(x), 607 

wedge product, see vector product 

weight 
of relative tensor, 964 
of variable, 477 

weight function, 555, 790 

Wiener—Kinchin theorem, 450 

WKB methods, 895-905 
accuracy, 902 
general solutions, 897 
phase memory, 895 
the Stokes phenomenon, 903 

work done 
by force, 381 
vector representation, 220 

Wronskian 
and Green’s functions, 527 
for second solution of ODE, 544, 580 
from ODE, 532 
test for linear independence, 491, 532 


X-ray scattering, 237 


Y/"(0, b), see spherical harmonics 
Y,(x), Bessel functions of second kind, 607 
Young’s modulus, 677, 953 


z, as a complex number, 84 
z*, as complex conjugate, 89-91 
zero (null) 
matrix, 254, 255 
operator, 249 
unphysical state |0), 650 
vector, 214, 242, 556 
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